[webkit-dev] Running W3C tests using wptserve

[youenn fablet]

2015-02-01 3:08 GMT+01:00 Benjamin Poulain <benjamin at webkit.org>:
> Hi Youenn,
>
> Thanks for working on this.
>
> How do we run the imported tests? Do we just use run-webkit-test passing
> 'imported/w3c'?

Yes.

> Is there a way to get the percentage of W3C tests that succeed? It would be
> useful to have a report per directory so that we improve it over time until
> we reach 100%.

That is part of another tool: https://bugs.webkit.org/show_bug.cgi?id=134766
It basically uses run-webkit-test on the whole W3C test suite (or a subset).
It checks conformance by searching for PASS/FAIL assertions.
You can find an old version of the report here:
http://youennf.github.io/w3c-reports/12092014/w3c_conformance_results.html

That tool would also be used to discriminate which tests to import.
https://bugs.webkit.org/show_bug.cgi?id=134767 would then be used to
refresh LayoutTests/imported/w3c (new tests, -expected.txt
creation/refresh)

I plan to write a page on the wiki when these things will start to
appear on the trunk.

Any suggestion/help most welcome.

>
> Benjamin
>
>
> On 1/31/15 11:57 AM, youenn fablet wrote:
>>
>> Hi all,
>>
>> W3C wptserve server is now used to serve all tests under
>> LayoutTests/imported/w3c/web-platform-tests.
>>
>> Currently, only two tests are run within that folder:
>> - web-platform-tests/domparsing/DOMParser-parseFromString-html.html
>> - web-platform-tests/domparsing/insert-adjacent.html
>>
>> Should there be any issus with those tests, the problem may be related
>> to running wptserve.
>> The wptserve server log is dumped in the the layout-test-results folder.
>> Please let me know if you see any such issue, particularly on WebKit bots.
>>
>> Next planned steps in that area are:
>> 1. Update Tools/Scripts/import-w3c-tests according this new set-up
>> 2. Migrate already imported W3C WPT tests within
>> LayoutTests/imported/w3c/web-platform-tests
>> 3. Further improve the tools to make import of new tests and re-import
>> existing tests as easy as possible
>>
>> Any suggestion welcome.
>>
>> Thanks,
>>     youenn
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev