[Webkit-unassigned] [Bug 97112] REGRESSION (r126189): No more mismatch-did-not-occur failures in reftests

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=97112





--- Comment #3 from Dirk Pranke <dpranke at chromium.org>  2012-09-19 13:53:01 PST ---
(In reply to comment #2)
> (In reply to comment #1)
> > I'm a bit confused ... that log message says that although the hashes were the same, the images were different, and so there was a mismatch (hence the test ran "as expected").
> > 
> > That seems to be the opposite of what you're saying?
> 
> Indeed the hashes are the same (as they should be) but the image of the reference driver output is None because DumpRenderTrees don't seem to output diff images when the hashes match. The Port.diff_image method returns a (True, 0, None) tuple when either image content is missing[1], and this is considered as a difference between reference and actual output in the current SingleTestRunner._compare_output_with_reference code, meaning the test in question passes (mismatch occurred) while it should fail.
> 

Hm. Okay, well, we seem to have two choices. We can either (1) not pass the hash to the reference test as input (so that we always generate both PNGs) or (2) assume that if the hashes match we don't need to actually diff the two images.

I'm inclined to do the former, since we've seen problems with hashes before, and it's not clear if there's any real perf impact to getting the second PNG.

FWIW, though, both chromium linux and apple mac also produce different hashes for the two inputs.


> > 
> > FWIW, I just ran this on chromium mac (on lion) and the hashes were actually different. I am rebuilding the apple-mac port to check there.
> 
> The hashes shouldn't differ, though. Don't know what's happening there.
> 
> 
> [1] http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/layout_tests/port/base.py#L346

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.