[Webkit-unassigned] [Bug 180420] WebDriver: add support for test expectations

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=180420

--- Comment #5 from Carlos Garcia Campos <cgarcia at igalia.com> ---
I've taken a look at pytest-expect and I have a few comments:

 - It could be a good idea to mark tests individually on collection no matter what format we use for the expectations file. I personally prefer to use a simpler format than layout tests expectations, with all the expectations in a single file.

 - I don't think we need to fork and modify any third-party plugin, pytest-expect in this case. Most of the code is actually to parse and write expectations in a file format we don't want. So, we just need our own plugin using pytest_collection_modifyitems to mark tests on collection.

 - However, it's not currently possible to use this approach for w3c tests, because wptrunner records XPASS as PASSED and SKIPPED as ERROR ("In-test skip decorators are disallowed, please use WPT metadata to ignore tests."). So, we would need to use our own record plugins (or override the wptrunner one) and call pytest ourselves like we do for the selenium tests.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20171214/bf59d877/attachment.html>