[Webkit-unassigned] [Bug 97112] REGRESSION (r126189): No more mismatch-did-not-occur failures in reftests

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=97112





--- Comment #4 from Zan Dobersek <zandobersek at gmail.com>  2012-09-20 10:24:55 PST ---
(In reply to comment #3)
> 
> Hm. Okay, well, we seem to have two choices. We can either (1) not pass the hash to the reference test as input (so that we always generate both PNGs) or (2) assume that if the hashes match we don't need to actually diff the two images.
> 
> I'm inclined to do the former, since we've seen problems with hashes before, and it's not clear if there's any real perf impact to getting the second PNG.
> 

I'll upload a patch that doesn't pass the hash to driver input.

On a related note, this would be easier to spot if the test was reported as unexpectedly passing. But the test is reported as such only if pixel testing is enabled, despite the test itself being a reftest. Also discovered is the following problem - the expectation for the failing test is not taken into account (the failure is not expected) if in the following format:
fast/harness/sample-fail-mismatch-reftest.html [ ImageOnlyFailure ]

The following two expectations seem to work, though:
(1) fast/harness/sample-fail-mismatch-reftest.html [ ImageOnlyFailure WontFix ]
(2) Bug(GTK) fast/harness/sample-fail-mismatch-reftest.html [ ImageOnlyFailure ]

I'll open two separate bugs for these issues.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.