[webkit-dev] Pixel test experiment

[Nikolas Zimmermann]

Am 08.10.2010 um 20:14 schrieb Jeremy Orlow:

> I'm not an expert on Pixel tests, but my understanding is that in  
> Chromium (where we've always run with tolerance 0) we've seen real  
> regressions that would have slipped by with something like tolerance  
> 0.1.  When you have 0 tolerance, it is more maintenance work, but if  
> we can avoid regressions, it seems worth it.

Well, that's why I initially argued for tolerance 0. Especially in SVG  
we had lots of regressions in the past that were below the 0.1  
tolerance. I fully support --tolerance 0 as default.

Dirk & me are also willing to investigate possible problem sources and  
minimize them.
Reftests as Simon said, are a great thing, but it won't help with  
official test suites like the W3C one - it would be a huge amount of  
work to create reftests for all of these...

Cheers,
Niko