[webkit-dev] Importing W3C tests to webkit

[Ryosuke Niwa]

As I have said in the past, we should just import all tests, and treat
non-text, non-ref tests as pixel tests. If we wanted to reduce the number
of pixel tests we import, then we should submit those patches to W3C
instead of directly submitting them to WebKit.

In general, I don't buy the argument that adding more pixel tests incurs
more cost than the benefit we get from importing the tests. If that were
the case, we can just get rid of the existing pixel tests we have.

The only sane argument I've heard so far to gate pixel tests is that the
correctness of such tests need to be manually inspected, which requires a
lot of manual labor and is very error prone.

For script-based tests that print PASS/FAIL, it seems better to just check
in *-expected-failure.txt or even -expected.txt files because we'll see
PASS/FAIL in the expected tests themselves.

- Ryosuke

On Wed, May 23, 2012 at 10:26 AM, Jacob Goldstein <jacobg at adobe.com> wrote:

> At the WebKit contributor's meeting in April, we discussed a process for
> importing third party tests into the WebKit repository (specifically, from
> the W3C test repository).
>
> I documented the process that we came up with at the meeting on the WebKit
> wiki, here:
> http://trac.webkit.org/wiki/ImportingThirdPartyTests
>
> It would be great if we could get some feedback on this process.
>
> Once we can finalize the details for how this should function, the effort
> can proceed - scripts can be written to help automate the effort and an
> actual import of select W3C test suites can be performed.
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120523/40ccf911/attachment.html>