[webkit-dev] Upstreaming from LayoutTests to web-platform-tests, coordinating Blink+WebKit

[Maciej Stachowiak]

> On Nov 17, 2017, at 7:05 AM, Frédéric WANG <fred.wang at free.fr> wrote:
> 
> Hi Philip,
> 
> We (at Igalia) are strong supporters of WPT tests and we've appreciated a lot the effort made by Youenn and others to make them possible in WebKit. They are very important for interoperability and make life much easier for developers working on different WebKit ports and different Web engines.
> 
> Ideally, we believe that future conformance tests in WebKit should be written using the WPT format and submitted to web-platform-tests repository (of course there are exceptions when the WPT API is too limited). WPT tests have references to spec URLs and their own git history, so it should always be possible to know why they have been written. This also reduces review cost on WebKit's side, as test review can be done by experts in the relevant specs.

WebKit has a lot of tests that were regression tests for a specific bug fix, not as conformance tests (though they might b useful for that too). In such cases, the  association with the bugs.webkit.org bug is more important than the spec URL. That’s particularly the case when the test has been changed multiple times to reflect further WebKit behavior changes. I know that I’ve personally found it very useful to look at revision history of tests, or search for bug numbers or keywords in LayoutTests/ChangeLog to find related tests.

Not saying this is the sole purpose of tests, but losing this ability or making it more awkward would be a downside to deleting tests from the WebKit repo.

> 
> Probably it would be nice to convert old tests to WPT and check duplication to avoid increasing runtime cost but that's indeed going to be a big effort. It could also be possible to improve how the CI tests are executed in the WebKit project, taking inspiration from Chromium or Mozilla projects (i.e. run all tests before a patch lands in the main repo, but allowing devs to run a relevant subset during development).

Maybe I’m missing something, but isn’t that already how it works in WebKit? EWS and buildbot run all the tests, but on your own machine you can get run-webkit-tests to run any subset you want.

> 
> In any case, Igalia definitely supports this proposal and is happy to help where possible.
> 
> Frédéric, for Igalia's Web Platform Team
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev