[webkit-dev] Importing W3C tests to webkit

[Jacob Goldstein]

I agree.  If nothing else, getting W3C tests into the WebKit repository will help catch regressions.

From: Ryosuke Niwa <rniwa at webkit.org<mailto:rniwa at webkit.org>>
To: Jacob Goldstein <jacobg at adobe.com<mailto:jacobg at adobe.com>>
Cc: WebKit Development <webkit-dev at lists.webkit.org<mailto:webkit-dev at lists.webkit.org>>
Subject: Re: [webkit-dev] Importing W3C tests to webkit

As I have said in the past, we should just import all tests, and treat non-text, non-ref tests as pixel tests. If we wanted to reduce the number of pixel tests we import, then we should submit those patches to W3C instead of directly submitting them to WebKit.

In general, I don't buy the argument that adding more pixel tests incurs more cost than the benefit we get from importing the tests. If that were the case, we can just get rid of the existing pixel tests we have.

The only sane argument I've heard so far to gate pixel tests is that the correctness of such tests need to be manually inspected, which requires a lot of manual labor and is very error prone.

For script-based tests that print PASS/FAIL, it seems better to just check in *-expected-failure.txt or even -expected.txt files because we'll see PASS/FAIL in the expected tests themselves.

- Ryosuke

On Wed, May 23, 2012 at 10:26 AM, Jacob Goldstein <jacobg at adobe.com<mailto:jacobg at adobe.com>> wrote:
At the WebKit contributor's meeting in April, we discussed a process for importing third party tests into the WebKit repository (specifically, from the W3C test repository).

I documented the process that we came up with at the meeting on the WebKit wiki, here:
http://trac.webkit.org/wiki/ImportingThirdPartyTests

It would be great if we could get some feedback on this process.

Once we can finalize the details for how this should function, the effort can proceed - scripts can be written to help automate the effort and an actual import of select W3C test suites can be performed.

_______________________________________________
webkit-dev mailing list
webkit-dev at lists.webkit.org<mailto:webkit-dev at lists.webkit.org>
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120523/06c899c8/attachment.html>