[Webkit-unassigned] [Bug 180420] WebDriver: add support for test expectations

bugzilla-daemon at webkit.org bugzilla-daemon at webkit.org
Tue Jan 16 08:49:58 PST 2018


https://bugs.webkit.org/show_bug.cgi?id=180420

--- Comment #12 from Brian Burg <bburg at apple.com> ---
(In reply to Carlos Garcia Campos from comment #5)
> I've taken a look at pytest-expect and I have a few comments:
> 
>  - It could be a good idea to mark tests individually on collection no
> matter what format we use for the expectations file. I personally prefer to
> use a simpler format than layout tests expectations, with all the
> expectations in a single file.
> 
>  - I don't think we need to fork and modify any third-party plugin,
> pytest-expect in this case. Most of the code is actually to parse and write
> expectations in a file format we don't want. So, we just need our own plugin
> using pytest_collection_modifyitems to mark tests on collection.
> 
>  - However, it's not currently possible to use this approach for w3c tests,
> because wptrunner records XPASS as PASSED and SKIPPED as ERROR ("In-test
> skip decorators are disallowed, please use WPT metadata to ignore tests.").
> So, we would need to use our own record plugins (or override the wptrunner
> one) and call pytest ourselves like we do for the selenium tests.

I agree on your points. I'm currently pursuing the last task myself, with mixed results so far. I'll let you know if I figure it all out!

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20180116/04cc5b16/attachment.html>


More information about the webkit-unassigned mailing list