[webkit-dev] Pixel test experiment
zimmermann at physik.rwth-aachen.de
Fri Oct 8 00:46:41 PDT 2010
Am 08.10.2010 um 00:44 schrieb Maciej Stachowiak:
> On Oct 7, 2010, at 6:34 AM, Nikolas Zimmermann wrote:
>> Good evening webkit folks,
>> I've finished landing svg/ pixel test baselines, which pass with --
>> tolerance 0 on my 10.5 & 10.6 machines.
>> As the pixel testing is very important for the SVG tests, I'd like
>> to run them on the bots, experimentally, so we can catch
>> regressions easily.
>> Maybe someone with direct access to the leopard & snow leopard
>> bots, could just run "run-webkit-tests --tolerance 0 -p svg" and
>> mail me the results?
>> If it passes, we could maybe run the pixel tests for the svg/
>> subdirectory on these bots?
> Running pixel tests would be great, but can we really expect the
> results to be stable cross-platform with tolerance 0? Perhaps we
> should start with a higher tolerance level.
Sure, we could do that. But I'd really like to get a feeling, for
what's problematic first. If we see 95% of the SVG tests pass with --
tolerance 0, and only a few need higher tolerances
(64bit vs. 32bit aa differences, etc.), I could come up with a per-
file pixel test tolerance extension to DRT, if it's needed.
How about starting with just one build slave (say. Mac Leopard) that
runs the pixel tests for SVG, with --tolerance 0 for a while. I'd be
happy to identify the problems, and see
if we can make it work, somehow :-)
More information about the webkit-dev