[Webkit-unassigned] [Bug 174504] New: Marking a directory as [ Pass ] or [ Failure ] or [ Crash ] should not override test-specific expectations

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=174504

            Bug ID: 174504
           Summary: Marking a directory as [ Pass ] or [ Failure ] or [
                    Crash ] should not override test-specific expectations
           Product: WebKit
           Version: WebKit Nightly Build
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: jbedard at apple.com
                CC: lforschler at apple.com

Currently, when we mark an entire directory as something, we simply recursively mark every test in that directory as Pass/Failure/Crash/Skip.  We should be smarter about this, particularly with 'Pass.'  For example:

TestExpectations: edit/mac [ Skip ]
platform/wk2/TestExpectations: edit/mac/some-test.html [ Failure ] # XYZ is not implemented in WK2
platform/mac/TestExpectations: edit/mac [ Pass ] # Now edit/mac/some-test.html is marked as passing

This results in 1) duplicated expectations, 2) bugs separated across expectation files and 3) it being difficult to garden tests since platforms frequently mark entire directories as [ Pass ] when those platforms support a feature which their more general parent platform does not.  (ie: Mac generally may not support a given feature, but Sierra might).

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20170714/574f558b/attachment.html>