[webkit-dev] Pixel test experiment

[Nikolas Zimmermann]

Am 08.10.2010 um 00:44 schrieb Maciej Stachowiak:

>
> On Oct 7, 2010, at 6:34 AM, Nikolas Zimmermann wrote:
>
>> Good evening webkit folks,
>>
>> I've finished landing svg/ pixel test baselines, which pass with -- 
>> tolerance 0 on my 10.5 & 10.6 machines.
>> As the pixel testing is very important for the SVG tests, I'd like  
>> to run them on the bots, experimentally, so we can catch  
>> regressions easily.
>>
>> Maybe someone with direct access to the leopard & snow leopard  
>> bots, could just run "run-webkit-tests --tolerance 0 -p svg" and  
>> mail me the results?
>> If it passes, we could maybe run the pixel tests for the svg/  
>> subdirectory on these bots?
>
> Running pixel tests would be great, but can we really expect the  
> results to be stable cross-platform with tolerance 0? Perhaps we  
> should start with a higher tolerance level.

Sure, we could do that. But I'd really like to get a feeling, for  
what's problematic first. If we see 95% of the SVG tests pass with -- 
tolerance 0, and only a few need higher tolerances
(64bit vs. 32bit aa differences, etc.), I could come up with a per- 
file pixel test tolerance extension to DRT, if it's needed.

How about starting with just one build slave (say. Mac Leopard) that  
runs the pixel tests for SVG, with --tolerance 0 for a while. I'd be  
happy to identify the problems, and see
if we can make it work, somehow :-)

Cheers,
Niko