[webkit-dev] Upstreaming from LayoutTests to web-platform-tests, coordinating Blink+WebKit

[Frédéric WANG]

On 17/11/2017 17:23, Maciej Stachowiak wrote:
> I should also note that pulling in the test suite won't automatically
> get the bugs fixed or prioritized, or even filed in the bug tracker
> necessarily.eratwerATWeRTWAerATWertwAerer 
Sure. However, I believe in general having more people importing the WPT
tests and trying to match the behavior of the spec and of other browsers
would help, if only to be aware of issues (as Philip mentioned we could
also improve notifications etc). In the particular cases I mentioned
https://bugs.webkit.org/show_bug.cgi?id=162668 was reported one year ago
after Khaled Hosny (invited expert for the WebFonts WG) fixed the bug in
the GTK port while https://bugs.webkit.org/show_bug.cgi?id=179237 just
requires to backport the changes in Chromium to match the latest WebVTT
spec. These are of course only two examples, but in general more tests
shared with the rest of the Web Platform community (developers and spec
authors) are very useful for improving WebKit and interoperability.
> I find it super convenient that EWS runs all the tests even on WIP
> patches. It often catches test failures in tests I didn't think to run
> myself. I think it would be great if EWS could run tests on more
> platforms. Just running release regression tests on more platforms
> would be a big win, if debug builds are the gating factor. I guess it
> would be kind of neat to have a feature of "run the tests across all
> configurations, but only some of them". But I don't think I would
> prioritize it over having more kinds of tests, or finding ways to make
> the full test suite run faster.
Right, running all the tests is nice. But sometimes I personally feel
guilty to have to execute everything on EWS for example to retrieve
results for macOS/iOS (when I did not have access to a mac) or to fix
compilation failures on Windows ports. So better granularity would be
helpful in my opinion. In any cases, this was just one idea and people
working on CI and infrastructure are in better position to know how to
improve running time of WPT tests. I'm surprised that I don't hear such
complaints from Mozilla and Chromium (but maybe I'm not aware of them).

-- 
Frédéric Wang - frederic-wang.fr


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20171124/ac89ac52/attachment.bin>