[webkit-dev] Another WPT bite

[Joseph Pecoraro]

> On May 12, 2017, at 2:39 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:
> 
> On Fri, May 12, 2017 at 12:04 PM, Alexey Proskuryakov <ap at webkit.org <mailto:ap at webkit.org>> wrote:
> 
>> We don't have a concept of "first class", but I hope that when choosing
>> between looking into a simple test that was added for a known important bug,
>> and looking into an imported test whose importance is unclear, any WebKit
>> engineer will pick the former. And since no one can fix all the things, such
>> prioritization makes imported tests less effective.
> 
> This is absolutely not how I operate at all. Since almost all custom
> elements and shadow DOM API tests I wrote are written using
> testharness.js and upstreamed to web-platform-tests, they have been
> deleted from LayoutTests/fast/shadow-dom &
> LayoutTests/fast/custom-elements.
> 
> As such, if any new shadow DOM or custom elements tests under
> LayoutTests/imported/ start to fail, then we must fix them since we
> idon'thave any other test coverage.


On the topic of LayoutTest/imported tests, can someone describe the current process of working with LayoutTest/imported?

How do we handle a broken test in our tree?

• Do we modify our expectations?
	- If so, how do we remember to change the expectations in a later import?

• Do we modify the test? More generally, are any modifications allowed inside of LayoutTest/imported?
	- If so, how do we track the fact that we have modifications so that a later import doesn't just wipe out our changes?
	- Without tooling, is the person who modifies the test responsible for making an upstream pull request?

How do we handle a new imported test that WebKit fails?

To avoid bots being red this test will end up getting skipped or marked as flakey.

How do we ensure this actually gets addressed and not ignored / forgotten about?

To make upstream changes someone has to make a pull-request:

I've had some upstream patches reviewed immediately and others are still in review after 3+ months.

In an earlier webkit-dev email Youenn mentioned:
https://lists.webkit.org/pipermail/webkit-dev/2017-April/028932.html

> As done by other browser teams, a test that is reviewed in WebKit bugzilla and
> landed in WebKit can be readily merged into WPT without additional review
> (although additional review is always great!).


I don't have commit access to the WPT GitHub. So ultimately I still have to go through another review to get my pull requests accepted.

- Joe

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20170512/f6cc5858/attachment.html>