[webkit-dev] Supporting <link rel=> for finding ref tests

[Frédéric Wang]

On 12/11/2019 20:06, Simon Fraser wrote:
>> On Nov 12, 2019, at 4:52 AM, Frédéric Wang <fwang at igalia.com> wrote:
>>
>> On 09/11/2019 04:02, Ryosuke Niwa wrote:
>>>>>>  - Requires us modifying each port's DRT to support this format
>>>>>>
>>>>>> No, it just requires webkitpy hacking which I've done in the patch.
>>>>> I'm not certain writing a bunch of regular expressions in webkitpy is
>>>>> a reliable mechanism to find expected results. Another issue I found
>>>>> back then was that it significantly slowed run-webkit-tests' startup
>>>>> time because WPT has a workflow to find all tests & their expected
>>>>> results upfront before any tests could run.
>>>> The patch uses html5lib (via BeautifulSoup), which is exactly what WPT, and our importer use to find the ref tests.
>>>>
>>>> We don't find references up-front; only when running each test. This patch does add some overhead for parsing each test file,
>>>> which I measured to be about 1-2 sec on a directory which took 30s to run. I think this slight slowdown is worthwhile (we could
>>>> probably eliminate it with some webkitpy optimizations).
>>> Hm... that's ~3% overhead.
>> @Simon: I agree with Ryosuke that 3% sounds big. IIUC you are parsing
>> the HTML file when running each test? I thought that there is a
>> MANIFEST.json file which is supposed to cache that information, why
>> can't we use it?
> I don't see any files call MANIFEST.json in the wpt repo.

This is a generated file:
https://github.com/web-platform-tests/wpt/blob/7c50c216081d6ea3c9afe553ee7b64534020a1b2/tools/runner/update_manifest.py#L12

-- 
Frédéric Wang