[webkit-dev] How to deal with 1-pixel differences in reftests ?

[Simon Fraser]

> On Nov 18, 2015, at 9:04 AM, Darin Adler <darin at apple.com> wrote:
> 
>> On Nov 18, 2015, at 4:36 AM, Carlos Alberto Lopez Perez <clopez at igalia.com> wrote:
>> 
>> Some reference tests give a 1-pixel or very few pixel differences [1].
> 
> Why? We need to understand why.
> 
>> Should we tolerate a few pixel differences for reftests ?
> 
> I don’t think we should. I think we should look more deeply into why these differences exist and consider writing the reference tests differently. We should not add tolerance when we don’t yet know why the results are different.

There are some well-understood reasons why a test might not exactly match its reference. One is that the test uses compositing layers to do clipping, but the reference just clips with drawing, and these are not expected to give a pixel-perfect match.

I proposed a way to allow a test to specify a custom tolerance in https://bugs.webkit.org/show_bug.cgi?id=149828. If we had this, it would allow me to fix 142258 <https://bugs.webkit.org/show_bug.cgi?id=142258> and 146523 <https://bugs.webkit.org/show_bug.cgi?id=146523>.

Simon

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.webkit.org/pipermail/webkit-dev/attachments/20151118/9a994abb/attachment.html>