[Webkit-unassigned] [Bug 231265] New: [EWS] Allow the optimization of running only the subset of failed tests on run-layout-tests-without-patch also for patches modifying the TestExpectations files

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=231265

            Bug ID: 231265
           Summary: [EWS] Allow the optimization of running only the
                    subset of failed tests on
                    run-layout-tests-without-patch also for patches
                    modifying the TestExpectations files
           Product: WebKit
           Version: WebKit Nightly Build
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: clopez at igalia.com
                CC: aakash_jain at apple.com, ap at webkit.org

On bug 219500 an optimization for run-layout-tests-without-patch was implemented, but this optimization was avoided for the following corner case:

> Consider a patch that removes a [ Skip ] test expectation. If the test still fails in reality, then EWS will be green anyway! That's because all three runs will see it fail - passing the test on command line overrides [ Skip ].

However, instead of avoiding the optimization in this case we can pass the flag '--skipped=always' to run-webkit-tests.

That way the test will not be run if it is marked as skipped, even when it is specified on the command-line. So it is safe to apply the optimization in that case as well.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20211005/7269a759/attachment.htm>