[Webkit-unassigned] [Bug 55909] New: new-run-webkit-tests --run-singly option is busted

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=55909

           Summary: new-run-webkit-tests --run-singly option is busted
           Product: WebKit
           Version: 528+ (Nightly build)
          Platform: PC
        OS/Version: Mac OS X 10.5
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
        AssignedTo: webkit-unassigned at lists.webkit.org
        ReportedBy: jamesr at chromium.org


When running with --run-singly option tests that fail and are expected to fail are producing 'unexpected pass' output.  Looking at the actual output from the test they seem to be really failing.

For example, canvas/philip/tests/2d.composite.operation.clear.html is marked as an expected TEXT diff and actually does fail with a TEXT diff.  When run with --run-singly --verbose on chromium linux I get this output:

...
2011-03-07 15:32:45,836 13188 executive.py:365 DEBUG "/usr/bin/wdiff --start-delete=##WDIFF_DEL## --end-delete=##WDIFF_END## --start-insert=##WDIFF_ADD## --end-insert=##WDIFF_END## /usr/local/google/home/jamesr/chrome/src/webkit/Release/layout-test-results/canvas/philip/tests/2d.composite.operation.clear-expected.txt /usr/local/google/home/jamesr/chrome/src/webkit/Release/layout-test-results/canvas/philip/tests/2d.composite.operation.clear-actual.txt" took 0.01s
2011-03-07 15:32:45,883 13188 executive.py:365 DEBUG "ruby -I /usr/local/google/home/jamesr/chrome/src/third_party/WebKit/Websites/bugs.webkit.org/PrettyPatch /usr/local/google/home/jamesr/chrome/src/third_party/WebKit/Websites/bugs.webkit.org/PrettyPatch/prettify.rb /usr/local/google/home/jamesr/chrome/src/webkit/Release/layout-test-results/canvas/philip/tests/2d.composite.operation.clear-diff.txt" took 0.05s
2011-03-07 15:32:45,989 13188 worker_mixin.py:141 DEBUG worker/0 canvas/philip/tests/2d.composite.operation.clear.html passed
2011-03-07 15:32:45,990 13184 printing.py:541 INFO   canvas/philip/tests/2d.composite.operation.clear.html -> unexpected pass
2011-03-07 15:32:45,990 13188 worker.py:83 DEBUG worker/0 done
...

but if I diff the output on disk locally...

$ diff /usr/local/google/home/jamesr/chrome/src/webkit/Release/layout-test-results/canvas/philip/tests/2d.composite.operation.clear-expected.txt /usr/local/google/home/jamesr/chrome/src/webkit/Release/layout-test-results/canvas/philip/tests/2d.composite.operation.clear-actual.txt
1c1
< Passed
---
> Failed assertion ctx.globalCompositeOperation == 'xor' (got clear[string], expected xor[string])

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.