[webkit-dev] Supporting <link rel=> for finding ref tests

[Simon Fraser]

I'd like to land a patch to support finding test references via <link rel="match/mismatch">:
https://bugs.webkit.org/show_bug.cgi?id=203784 <https://bugs.webkit.org/show_bug.cgi?id=203784>

There has been some discussion about this in the past:
https://lists.webkit.org/pipermail/webkit-dev/2011-November/018470.html <https://lists.webkit.org/pipermail/webkit-dev/2011-November/018470.html>

But I think the benefits outweigh the drawbacks. As that mail states:
> *Link element approach*
> Pros:
> 
>    - Can reuse same ref. file for multiple tests
Still true.
>    - Can have multiple ref. files for single test
True but no something that we support, and I haven't see any WPT use this (our importer throws an error if it sees this)
>    - Information is self-contained in the test file
Still true
>    - We may get away with test suite build step
It certainly simplifies WPT test import.
Currently importing some CSS suites (e.g. css-backgrounds) results in broken -expected.html files because copying them breaks references to sub resources.
> (It turns out that we can't convert W3C ref tests to use WebKit conventions
> due to the first two points.)
We're doing this much more now, and the "multiple references" point is moot, so I think we can import WPT tests mostly as-is.
> 
> Cons:
> 
>    - Requires us modifying each port's DRT to support this format
No, it just requires webkitpy hacking which I've done in the patch.
>    - Adding link elements itself may affect tests (all W3C tests are
>    required to have link elements at the moment)
I haven't seen this be an issue.
>    - Hard to understand relationship between files. e.g. if we want to
>    figure out which tests use ref.html, we must look at all test files
This is true, but I don't really see it being a problem in practice. What I have seen is us importing CSS 2.1 tests that have foo.html and foo-ref.html, and treating foo-ref.html as a test so generating foo-expected.txt and foo-ref-expected.txt. That seems worse.
>    - Other browser vendors (Firefox and Opera) prefer manifest file approach
This is no longer true. "reftest.list" files are deprecated (webkit.org/b/203783, https://github.com/web-platform-tests/wpt/issues/20060 <https://github.com/web-platform-tests/wpt/issues/20060>).
So now that WPT is heavily invested in <link rel=> I think we should follow suite. It will simplify WPT import, and reduced the number of cloned -expected.html files significantly.
Simon

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20191108/f3f93124/attachment.htm>