[webkit-dev] How to deal with 1-pixel differences in reftests ?

[Carlos Alberto Lopez Perez]

Hi,

Some reference tests give a 1-pixel or very few pixel differences [1].

I'm not sure if this really indicates a problem in the WebKit code, or
it indicates that we are just too strict not allowing even a very small
percentage in pixel differences for this kind of tests.

Should we tolerate a few pixel differences for reftests ?

I have done some tests, and the test in [1] passes for any value of
tolerance >= 0.00001% (with the GTK port).

I'm inclined to allow a very small value, for example a 0.001% (that
would be 100 times stricter than the tolerance value we use for the
other tests)


Regards
-------

[1]

For example, this is happening in the GTK port:
https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20%28Tests%29/r192415%20%2812243%29/imported/blink/fast/canvas/canvas-clip-stack-persistence-diffs.html
The diff image normalized (so you can see where is the diff):
http://people.igalia.com/clopez/wkbug/151261/canvas-clip-stack-persistence-diff_normalized.png