[Webkit-unassigned] [Bug 73978] New: Ease specifying narrow test_expectations lines for multiple failures on same test

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=73978

           Summary: Ease specifying narrow test_expectations lines for
                    multiple failures on same test
           Product: WebKit
           Version: 528+ (Nightly build)
          Platform: Unspecified
        OS/Version: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
        AssignedTo: webkit-unassigned at lists.webkit.org
        ReportedBy: scheib at chromium.org
                CC: dpranke at chromium.org


It is best to define a test expectation as narrowly as possible such that configurations that are succeeding continue to provide regression detection. This can be challenging when a test is failing in multiple ways. E.g. if debug versions are SLOW but all LEOPARD CPU-CG builders are crashing.

Either new-run-webkit-tests could tolerate overlapping expectations for tests, e.g.:
BUG LEOPARD : test.html = CRASH PASS
BUG MAC RELEASE : test.html = TIMEOUT PASS

Or tools such as the chrome flakiness dashboard could generate suggestions of how to narrowly define failures in non-overlapping ways.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.