[webkit-dev] Another WPT bite

[youenn fablet]

> On the topic of LayoutTest/imported tests, can someone describe the
> current process of working with LayoutTest/imported?
>
> How do we handle a broken test in our tree?
>
> • Do we modify our expectations?
>
> - If so, how do we remember to change the expectations in a later import?
>
>
If a test breaks, you are supposed to rebase the expected file.
A later import will rebase the expected file.

When importing, some subtests may not be passing.
This is ok, keep the expectation as is and do not add a FAIL line in
TestExpectations.


> • Do we modify the test? More generally, are any modifications allowed
> inside of LayoutTest/imported?
> - If so, how do we track the fact that we have modifications so that a
> later import doesn't just wipe out our changes?
> - Without tooling, is the person who modifies the test responsible for
> making an upstream pull request?
>
>
It is ok to modify the test only if you are making sure the test change is
upstreamed at landing time.
The author of the patch is currently responsible for upstreaming the
changes.
We will make things better in a not-too-far future.



> How do we handle a new imported test that WebKit fails?
>
> To avoid bots being red this test will end up getting skipped or marked as
> flakey.
>
> If it is flaky, mark it as such or mark it as skip
If it is consistently failing, it is doing some level of regression testing.
Another possibility is to not import it at all.
Please use LayoutTests/imported/w3c/resources/import-expectation.json for
that purpose.

How do we ensure this actually gets addressed and not ignored / forgotten
> about?
>
>
This is currently the responsibility of the developer doing the feature
work.
When a feature is complete, we should look at how much tests are passing
vs. skipped/failing.
This is conformance testing, we can use our own copy of WPT or W3C one.


>
> To make upstream changes someone has to make a pull-request:
>
>
> I've had some upstream patches reviewed immediately and others are still
> in review after 3+ months.
>
> In an earlier webkit-dev email Youenn mentioned:
> https://lists.webkit.org/pipermail/webkit-dev/2017-April/028932.html
>
> As done by other browser teams, a test that is reviewed in WebKit bugzilla
> and
>
> landed in WebKit can be readily merged into WPT without additional review
>
> (although additional review is always great!).
>
>
> I don't have commit access to the WPT GitHub. So ultimately I still have
> to go through another review to get my pull requests accepted.
>
>
You might get commit access.
Otherwise, some WebKit members have it and will do the cq for you.
Please ping me for that purpose. I am sure others might help too.

>
> - Joe
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20170512/79816a21/attachment.html>