[Webkit-unassigned] [Bug 224638] New: Migrate Python tests to pytest

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=224638

            Bug ID: 224638
           Summary: Migrate Python tests to pytest
           Product: WebKit
           Version: WebKit Nightly Build
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: gsnedders at apple.com
                CC: don.olmstead at sony.com, jbedard at apple.com,
                    webkit-bug-importer at group.apple.com
        Depends on: 224627

There has been occasional talk about moving our Python unit testing from the stdlib unittest library to pytest, which we already have for the sake of WPT's WebDriver tests.

pytest adds a variety of useful features (various ways to do test filtering, better debug integration, much better skip/xfail support, a de-facto standard parallel test-execution method (pytest-xdist), less boilerplate in tests), and avoids us having to reimplement much in our own wrapper around unittest.

One open question is whether we can break the specific arguments used by ./Tools/Scripts/test-webkitpy{,-python2} today?

Aside from that, there's a few more technical issues:

 * pytest's unittest support uses the stdlib unittest.TestLoader class, which we subclass in webkitpy.test.main, to optionally load methods beginning with serial_test_*, integration_test_*, and serial_integration_test_* as well as test_*; logically these would become marks in pytest

 * we have support for uploading test results (via --report) and our own JSON format (--json-output); we could either implement post-processing results from pytest or implement our own output format

 * pytest doesn't handle running tests in very different directories particularly well due to its rootdir and confdir handling (and for Source/WebKit/Scripts/webkit and Tools/Scripts/webkitpy the common root in the root of the WebKit repo, and we likely don't want to put any files there for pytest!); we could just have multiple invocations of pytest?

 * pytest-xdist doesn't support running a subset of tests sequentially (https://github.com/pytest-dev/pytest-xdist/issues/385), which we currently do for serial_* tests; this is another argument for multiple invocations of pytest


Referenced Bugs:

https://bugs.webkit.org/show_bug.cgi?id=224627
[Bug 224627] scm_unittest.py fails at import-time on Python 2
-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20210416/bdafcb78/attachment.htm>