[webkit-dev] Upstreaming from LayoutTests to web-platform-tests, coordinating Blink+WebKit

[Danyao]

Hi Philip,

Ali and I would be willing to help this wherever we can. It seems to us
that there may be a few different classes of tests that require varying
degree of effort to migrate. Do you have any estimates on how many tests
fall into each category, maybe based on your analysis of Blink LayoutTests?

   1. braindead to migrate
   - 1a) ref tests not using window.internals
      - 1b) tests using internals API that have trivial equivalent in
      testharness
   2. tests using non-trivial internals API so require some domain
   expertise in the functionality under test
      - 2a) priority features: maybe active development so more likely to
      diverge in the near future?
      - 2b) long tail
   3. tests that can't be migrated to WPT
      - 3a) tests that explicit use layerTreeAsText (tests in the
      compositing layer)
      - 3b) tests that dump render object tree (testing browser internals)


Thanks,
Danyao

On Wed, Nov 15, 2017 at 5:02 AM, Philip Jägenstedt <foolip at chromium.org>
wrote:

> Hello webkit-dev! (ecosystem-infra in Bcc)
>
> TPAC was last week, and there was much talk about web-platform-tests. Some
> notes are at https://lists.w3.org/Archives/Public/public-test-
> infra/2017OctDec/thread.html and in particular https://www.w3.org/
> 2017/11/07-testing-minutes.html.
>
> In Blink, we're thinking seriously about what it'd take to upstream large
> parts of LayoutTests into web-platform-tests, and we've realized that there
> are some risks here, beyond just making sure the tests are good and per
> spec. Specifically, one bad outcome would be if Blink successfully
> upstreamed thousands of tests that are also in WebKit, which then begin to
> diverge in small and large ways. That'd leave WebKit with more duplication
> between its own tests and web-platform-tests than any other engine, and a
> headache that grows over time.
>
> So, I think this would require close cooperation with the WebKit project.
> Some different ways this might happen:
>
>    1. A Blink developer and WebKit developer work together at the same
>    time to move their common tests in some area into web-platform-tests, each
>    removing identical tests that were upstreamed by the other and
>    consolidating what remains.
>    2. A Blink developer works to first upstream tests from WebKit (using
>    WebKit's coming export mechanism), waits for it to be imported into Blink,
>    and then removes the tests from Blink.
>    3. The reverse situation.
>
>
> Are there already some areas where the first approach might be doable,
> where the Blink and WebKit folks already know each other and are eager to
> do this? Is LayoutTests/css3/flexbox/ by any chance such a case?
>
> I realize it's hard to judge these approaches in the abstract, but am
> curious to hear any enthusiasm or concerns about it.
>
> Thanks!
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20171115/8936ce0a/attachment.html>