[webkit-dev] Upstreaming from LayoutTests to web-platform-tests, coordinating Blink+WebKit

[Philip Jägenstedt]

On Fri, Nov 17, 2017 at 5:23 PM, Maciej Stachowiak <mjs at apple.com> wrote:
>
>
>> On Nov 17, 2017, at 7:53 AM, Frédéric WANG <fred.wang at free.fr> wrote:
>>
>> On 17/11/2017 16:26, Maciej Stachowiak wrote:
>>> WebKit has a lot of tests that were regression tests for a specific
>>> bug fix, not as conformance tests (though they might b useful for that
>>> too). In such cases, the association with the bugs.webkit.org bug is
>>> more important than the spec URL. That’s particularly the case when
>>> the test has been changed multiple times to reflect further WebKit
>>> behavior changes. I know that I’ve personally found it very useful to
>>> look at revision history of tests, or search for bug numbers or
>>> keywords in LayoutTests/ChangeLog to find related tests.
>>> Not saying this is the sole purpose of tests, but losing this ability
or making it more awkward would be a downside to deleting tests from the
WebKit repo.
>> Well, I think we agree here, that's why "conformance" tests was
>> specified in my message. For such tests, connecting the history of tests
>> with the development of the specs is actually more important. For
>> example I recently noticed that some bugs with WOFF2 (on Apple's ports
>> only) and WebVTT have been ignored in WebKit because we don't
>> import/sync WPT tests for these specs.
>
> I think importing new test suites is a different question than
upstreaming/removing tests. In general importing more test suites is
probably good, but we probably need to tackle the WPT performance problem
before we pull in too many new WPT suites.
>
> I should also note that pulling in the test suite won't automatically get
the bugs fixed or prioritized, or even filed in the bug tracker necessarily.

This is the case in Chromium as well, although Robert Ma is working on
import notifications:
https://docs.google.com/document/d/1W3V81l94slAC_rPcTKWXgv3YxRxtlSIAxi3yj6NsbBw/edit?usp=sharing

We don't know yet what's going to work well, but I think that somehow,
before too long, we need to at least triage all new failures. Far from all
will be worth prioritizing of course, but sometimes the existence of the
test can reduce the effort and make it easier to prioritize.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20171124/fa370839/attachment.html>