[webkit-dev] Another WPT bite

[Ben Kelly]

On Fri, May 12, 2017 at 2:26 PM, Rick Byers <rbyers at chromium.org> wrote:

> On Fri, May 12, 2017 at 2:06 PM, Alexey Proskuryakov <ap at webkit.org>
> wrote:
>
>> Since imported WPT tests are very flaky, and are not necessarily written
>> to defend against important regressions, investigating issues with them is
>> relatively lower priority than investigating issues observed with WebKit
>> tests. So I would recommend not mixing tests for WebKit regressions with
>> WPT tests - if your test eventually ends up in LayoutTests/imported, it
>> will become a lot less effective.
>>
>
> FWIW this is absolutely NOT how we're treating this in chromium.  If this
> is how things end up in practice then we will have failed massively in this
> effort.
>
> We figure if we want the web to behave consistently, we really have no
> choice but to treat web-platform-tests as first class with all the
> discipline we give to our own tests.  As such we are actively moving
> <https://codereview.chromium.org/2877673004> many of our LayoutTests to
> web-platform-tests and depending entirely on the regression prevention they
> provide us from there.  Obviously there will be hiccups, but because our
> product quality will depend on web-platform-tests being an effective and
> non-flaky testsuite (and because we're starting to require most new
> features have web-platform-tests before they ship), I'm confident that
> we've got the incentives in place to lead to constant ratcheting up the
> engineering discipline and quality of the test suite.
>

FWIW, mozilla also treats WPT as first class tests.  We're not actively
moving old tests to WPT like google, but all new tests (at least in DOM)
are being written in WPT.  Of course, we do have exceptions for some tests
that require gecko-specific features (forcing GC, etc).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20170512/38d93057/attachment.html>