[webkit-dev] Exporting WPT tests

[Mike Pennisi]

Hi Youenn. My name is Mike, and I've been working with Google for the past 4
months or so to improve various aspects of the Web Platform Tests 
project (more
on that here [1]).

 > The only constraint I know of is that the test does not give flaky 
tests from
 > WPT Chrome/Firefox bots.

The full set of validation steps are described in the project's 
`.travis.yml`
file [2]. That's a bit tough to read even if you're familiar with 
TravisCI (we're
working on it!), but from WebKit's perspective, the only other relevant 
check
is for file linting. It's not very opinionated (mostly limited to objective
concerns) but still something to be aware of.

Also note that we're very close to including both Edge and Safari in the 
set of
browsers used to identify flaky tests! [3]

 > We do not have yet the tooling to automate the creation of a WPT 
GitHub PR
 > from a WebKit patch that lands.

I've recently been migrating tests for Service Workers from the Chromium
project to WPT. The process in place there is pretty slick. (Context for 
other
folks on the list: it's able to create commits that exclude 
Chromium-specific
files [4] and then submit GitHub pull requests from those, merging when CI
passes [5]. The patch Youenn mentioned is based on those files.)

I'm wondering if we can avoid duplicating effort by making a standalone 
tool.
It might even be the kind of thing we could host in the W3C GitHub
organization--whose to say that Edge (for example) wouldn't benefit from 
that,
too? I would love to be involved in that implementation.

But I'm getting ahead of myself :) I've CC'd Jeff Carp and Quinten 
Yearsley of
the Chromium team since they are currently working with that tooling.

So what do you folks think? Would it be practical to share code like this?

[1] https://bocoup.com/blog/diving-into-the-web-platform-tests
[2] https://github.com/w3c/web-platform-tests/blob/master/.travis.yml
[3] https://github.com/w3c/web-platform-tests/pull/5231
[4] 
https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/chromium_commit.py
[5] 
https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/wpt_github.py