[Webkit-unassigned] [Bug 252269] New: Match upstream WPT semantics for testharness tests

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=252269

            Bug ID: 252269
           Summary: Match upstream WPT semantics for testharness tests
           Product: WebKit
           Version: WebKit Nightly Build
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: gsnedders at apple.com
        Depends on: 161693, 252268, 161310

We know that a disproportionate amount of flakiness in LayoutTests comes from WPT; some degree of this is simply going to be the rate at which tests are added there versus the rest of LayoutTests.

But we also break the testharness API guarantees by expecting more from them by default: we expect what we log in the console to be constant (bug 161310), we expect assertion messages to be constant (bug 161693), and we expect the order of subtests to be constant (bug 252268).


Referenced Bugs:

https://bugs.webkit.org/show_bug.cgi?id=161310
[Bug 161310] Test infrastructure should allow to filter out text output before doing a comparison with the baseline
https://bugs.webkit.org/show_bug.cgi?id=161693
[Bug 161693] Some WPT testharness-based tests are flaky due to always-changing assertion failure messages
https://bugs.webkit.org/show_bug.cgi?id=252268
[Bug 252268] WPT testharness subtests should be unordered
-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20230214/aa904c60/attachment.htm>