[Webkit-unassigned] [Bug 80644] Test expectations and tolerance interfere

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=80644





--- Comment #2 from Tim Horton <timothy_horton at apple.com>  2012-03-09 13:07:15 PST ---
(In reply to comment #1)
> I don't think I understand what you're looking for ... the difference between an expected failure and an unexpected failure is that the former don't cause the test run itself to fail (since the tree goes red only on unexpected failures).
> 
> In your example, do you want css3/filters/crash-filter-change.html to turn the tree red if the pixel hash doesn't match? If so, why list it as an expected IMAGE failure at all?
> 
> Perhaps we need some sort of expectation of fuzzy pass but exact failure?

Yes, I think that's what we want. A fuzzy pass expectation (the overall result may be a pass because the tolerance can change, but the pixel hash will not pass).

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.