[webkit-dev] Standard process for exporting new WPT tests?

[Amal Hussein]

To elaborate on Youenn's point regarding wip improvements on *importing*
WPT:

The wpt import script is currently being updated with an enhanced git
workflow to identify renames, git moves, and deleted files. By default it
will output a report of changes, and when run with a *--update-files *flag,
it will update the webkit result files (snapshots, expectations, etc).

Additionally, the wpt test runner is being enhanced to support the
detection, and processing of all types of test result scenrios (new
failures, new passing results, etc). By default, a report of the changes
needed will be generated, and a flag will be used to implementation the
proposed changes. This supports both a local development workflow where
tests can be run without side effects, and a workflow where we can choose
to generate/update the webkit test result files.

The goal is to automate as much of the import workflow as possible, and
generate a patch with updates to exisiting result files, newly imported WPT
tests, newly created webkit result files.

In a future iteration, we plan to work with the infra team to open those
PRs on demand, or via chron job that would run once a day. We can either
use a keyword (ex: *wpt-sync-bot*)  or the bot user account to drive
permissions for who can merge, whether the bot should make a new PR or
update an open PR, etc. We have not fleshed out the specifics of the bot
patch process, but I hope you get a sense of the coming enhancements to
support the import workflow.


On Wed, May 23, 2018 at 10:37 AM, youenn fablet <youennf at gmail.com> wrote:

> I think WebKitten should be able to choose between 1 and 2 on a per patch
> basis.
>
> In both cases, Tools/Script/export-w3c-test-changes can be used to help
> upstreaming to WPT.
> The WPT exporter can push the changes to a branch in a GitHub WPT clone.
> It can also create the WPT PR on behalf of the WebKitten.
>
> As requested by some folks, integration of WPT exporter with webkit-patch
> is on its way (https://bugs.webkit.org/show_bug.cgi?id=184914)
>
> Any WPT PR that is reviewed downstream in WebKit can be merged without
> further WPT review.
> At the moment, given the WPT infra and the way we currently create WebKit
> PR, not every WebKitten is allowed to merge such PRs directly.
> Please cc/ping me in such a case.
>     Y
>
> Le mer. 23 mai 2018 à 02:41, Frédéric Wang <fwang at igalia.com> a écrit :
>
>> Hi all,
>>
>> When implementing new platform features in WebKit, some of us write new
>> WPT tests in order to verify spec conformance. One of the reason to use
>> this format is that the tests can then be easily shared with web engine
>> developers and people involved in standardization, an important feature
>> for web platform developers.
>>
>> Youenn has shared various ideas to export such WPT tests to the W3C
>> GitHub repository in the past [1] [2] [3] but I'm not sure the WebKit
>> community has reached an agreement.
>>
>> There are basically two ways to proceed:
>>
>> 1) Either submit WPT tests to GitHub (or Mozilla / Chromium) repository,
>> get them reviewed and do the import within the patch that implements the
>> feature in WebKit.
>>
>> 2) Or submit the patch with new WPT tests to WebKit, get it reviewed and
>> export it to GitHub (later sync with Mozilla / Chromium).
>>
>> The first option has the advantage to get more people involved in test
>> review, which sometimes gives faster replies or useful advice from
>> people who more familiar with the spec. However, the second option has
>> the advantage of not splitting the coding & testing parts and is better
>> for the WebKit implementers and reviewers. That latter option also
>> aligns with what Mozilla and Chromium developers do.
>>
>> Can we please agree on a way to proceed? And add documentation somewhere
>> for developers/reviewers?
>>
>> Thanks!
>>
>>
>> [1] https://lists.webkit.org/pipermail/webkit-dev/2017-March/028884.html
>> [2] https://lists.webkit.org/pipermail/webkit-dev/2017-April/028932.html
>> [3] https://lists.webkit.org/pipermail/webkit-dev/2017-
>> December/029837.html
>>
>> --
>> Frédéric Wang - frederic-wang.fr
>>
>>
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>>
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>


-- 
*Amal Hussein*
*Sr Open Web Engineer |* Bocoup
https://bocoup.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20180523/75905b8f/attachment.html>