[Webkit-unassigned] [Bug 63838] New: new-run-webkit-tests does not understand that mac uses test_expectations files

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=63838

           Summary: new-run-webkit-tests does not understand that mac uses
                    test_expectations files
           Product: WebKit
           Version: 528+ (Nightly build)
          Platform: Unspecified
        OS/Version: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
        AssignedTo: webkit-unassigned at lists.webkit.org
        ReportedBy: eric at webkit.org
                CC: abarth at webkit.org, ojan at chromium.org,
                    dpranke at chromium.org
            Blocks: 34984


new-run-webkit-tests does not understand that mac uses test_expectations files

When uses_expectations_file is false, these failures appear for mac:

 test     results    
+fast/canvas/webgl/gl-teximage.html    expected actual diff pretty diff    
+http/tests/cookies/third-party-cookie-relaxing.html    expected actual diff pretty diff    
+inspector/timeline/timeline-layout.html    expected actual diff pretty diff    
+plugins/npp-set-window-called-during-destruction.html    expected actual diff pretty diff    


Yet they're all mentioned in platform/mac/test_expectations.txt:

BUGWK58192 : plugins/npp-set-window-called-during-destruction.html = TEXT

// Flaky tests when run multi-process
BUGWK58192 : fast/dom/frame-loading-via-document-write.html = TEXT PASS
BUGWK58192 : http/tests/appcache/fail-on-update-2.html = TEXT PASS
BUGWK58192 : http/tests/appcache/fail-on-update.html = TEXT PASS
BUGWK58192 : http/tests/inspector/console-websocket-error.html = TEXT PASS
BUGWK58192 : fast/canvas/webgl/gl-teximage.html = TEXT PASS
BUGWK58192 : fast/frames/flattening/iframe-flattening-offscreen.html = TEXT PASS
BUGWK58192 : svg/dom/SVGScriptElement/script-set-href.svg = TEXT PASS


Part of the problem is in manager.py:

    # FIXME: If non-chromium ports start using an expectations file,
    # we should make this check more robust.
    results['uses_expectations_file'] = port_obj.name().find('chromium') != -1

however, I dont' think we want to set that bool as it does more than we want for mac.


Maybe we can't have our cake and eat it too.  WE should probably just move those to the Skipped list for Mac.  But platform/mac/test_expetations.txt was a useful tool for the ORWT->NRWT conversion as it's just tests which fail in NRWT but don't in ORWT.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.