[webkit-dev] Upstreaming Tests from WebKit to Web Platform Tests

[Philip Jägenstedt]

On Mon, Feb 6, 2017 at 7:32 PM Ryosuke Niwa <rniwa at webkit.org> wrote:

> On Mon, Feb 6, 2017 at 4:22 PM, youenn fablet <youennf at gmail.com> wrote:
>
> It seems we agree in moving this forward. Any objection?
> If so, let's start with small practical steps, something like a script to
> automatically generate WPT PR requests from a WebKit repo.
>
>
> I think we need to first agree on where to put new tests. If we're placing
> next to existing tests inside LayoutTests/imported/w3c/web-platform-tests/
> then identifying those new tests will be an issue.
>
> Also, there needs to be a mechanism to detect these new tests that have
> been merged into upstream when we're re-importing. It's possible for tests
> newly added in WebKit to be renamed, moved, or otherwise modified before we
> re-import them back.
>
> All these situations need to be carefully thought out first.
>

If it might be of any use, here's a doc where we hash out a lot of question
for 2-way sync in Chromium:
https://docs.google.com/document/d/1JgPTyIWmjlXyhyatiZ9A6fSbUQPjCyM26budEY90VDE/edit?usp=sharing

Linked from that is also an older doc with a lot of discussion:
https://docs.google.com/document/d/1JOcZsURB3ITsRtundqkpVVDrqo6x-_jdif7s_0rLJD0/edit?usp=sharing

When it comes to identifying which commits to export next, we look for the
most recent exported commit in web-platform-tests, map that to a Chromium
commit, and then inspect all later commits that touch
LayoutTests/external/wpt/. This scheme only works if commits are always
exported in order. Note also that commits before the most recent import can
also be candidates, because of imports racing with test changes in CQ.
Automatic imports are also blocked on all exports being finished, to not
(temporarily) revert changes.

Happy to provide more details on design and trade-offs if useful to you.
Regardless, I'm quite excited to see this happening in WebKit.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.webkit.org/pipermail/webkit-dev/attachments/20170207/2b488929/attachment.html>