[webkit-dev] Standard process for exporting new WPT tests?

[Ryosuke Niwa]

As we have preciously discussed, we should NEVER commit new tests into
LayoutTests/imported/w3c/web-platform-tests.

As long as that invariant is kept, I don't care where the test is reviewed.

- R. Niwa

On Wed, May 23, 2018 at 1:12 PM, Amal Hussein <amal at bocoup.com> wrote:

> To elaborate on Youenn's point regarding wip improvements on *importing*
> WPT:
>
> The wpt import script is currently being updated with an enhanced git
> workflow to identify renames, git moves, and deleted files. By default it
> will output a report of changes, and when run with a *--update-files *flag,
> it will update the webkit result files (snapshots, expectations, etc).
>
> Additionally, the wpt test runner is being enhanced to support the
> detection, and processing of all types of test result scenrios (new
> failures, new passing results, etc). By default, a report of the changes
> needed will be generated, and a flag will be used to implementation the
> proposed changes. This supports both a local development workflow where
> tests can be run without side effects, and a workflow where we can choose
> to generate/update the webkit test result files.
>
> The goal is to automate as much of the import workflow as possible, and
> generate a patch with updates to exisiting result files, newly imported
> WPT tests, newly created webkit result files.
>
> In a future iteration, we plan to work with the infra team to open those
> PRs on demand, or via chron job that would run once a day. We can either
> use a keyword (ex: *wpt-sync-bot*)  or the bot user account to drive
> permissions for who can merge, whether the bot should make a new PR or
> update an open PR, etc. We have not fleshed out the specifics of the bot
> patch process, but I hope you get a sense of the coming enhancements to
> support the import workflow.
>
>
> On Wed, May 23, 2018 at 10:37 AM, youenn fablet <youennf at gmail.com> wrote:
>
>> I think WebKitten should be able to choose between 1 and 2 on a per patch
>> basis.
>>
>> In both cases, Tools/Script/export-w3c-test-changes can be used to help
>> upstreaming to WPT.
>> The WPT exporter can push the changes to a branch in a GitHub WPT clone.
>> It can also create the WPT PR on behalf of the WebKitten.
>>
>> As requested by some folks, integration of WPT exporter with webkit-patch
>> is on its way (https://bugs.webkit.org/show_bug.cgi?id=184914)
>>
>> Any WPT PR that is reviewed downstream in WebKit can be merged without
>> further WPT review.
>> At the moment, given the WPT infra and the way we currently create WebKit
>> PR, not every WebKitten is allowed to merge such PRs directly.
>> Please cc/ping me in such a case.
>>     Y
>>
>> Le mer. 23 mai 2018 à 02:41, Frédéric Wang <fwang at igalia.com> a écrit :
>>
>>> Hi all,
>>>
>>> When implementing new platform features in WebKit, some of us write new
>>> WPT tests in order to verify spec conformance. One of the reason to use
>>> this format is that the tests can then be easily shared with web engine
>>> developers and people involved in standardization, an important feature
>>> for web platform developers.
>>>
>>> Youenn has shared various ideas to export such WPT tests to the W3C
>>> GitHub repository in the past [1] [2] [3] but I'm not sure the WebKit
>>> community has reached an agreement.
>>>
>>> There are basically two ways to proceed:
>>>
>>> 1) Either submit WPT tests to GitHub (or Mozilla / Chromium) repository,
>>> get them reviewed and do the import within the patch that implements the
>>> feature in WebKit.
>>>
>>> 2) Or submit the patch with new WPT tests to WebKit, get it reviewed and
>>> export it to GitHub (later sync with Mozilla / Chromium).
>>>
>>> The first option has the advantage to get more people involved in test
>>> review, which sometimes gives faster replies or useful advice from
>>> people who more familiar with the spec. However, the second option has
>>> the advantage of not splitting the coding & testing parts and is better
>>> for the WebKit implementers and reviewers. That latter option also
>>> aligns with what Mozilla and Chromium developers do.
>>>
>>> Can we please agree on a way to proceed? And add documentation somewhere
>>> for developers/reviewers?
>>>
>>> Thanks!
>>>
>>>
>>> [1] https://lists.webkit.org/pipermail/webkit-dev/2017-March/028884.html
>>> [2] https://lists.webkit.org/pipermail/webkit-dev/2017-April/028932.html
>>> [3] https://lists.webkit.org/pipermail/webkit-dev/2017-December/
>>> 029837.html
>>>
>>> --
>>> Frédéric Wang - frederic-wang.fr
>>>
>>>
>>> _______________________________________________
>>> webkit-dev mailing list
>>> webkit-dev at lists.webkit.org
>>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>>>
>>
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>>
>>
>
>
> --
> *Amal Hussein*
> *Sr Open Web Engineer |* Bocoup
> https://bocoup.com/
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20180523/b698b618/attachment.html>