[webkit-dev] Using ref tests for repaint bugs

[Julien Chaffraix]

>> I think this will be too fragile. It relies on the fact that a subsequent
>> paint won't obscure the repaint bug that you're trying to detect.
>>
>> I'm much rather we work towards detecting repaint bugs via dumping the
>> list of repaint rectangles. I don't think every test has to be a ref test.
>
>
> Don't the pixel tests face the same kind of risk, of a subsequent paint
> masking the repaint bug? Preserving this risk, but reducing the maintenance
> cost for the test is still an important improvement.

I second Simon's fragility argument. His solution is more viable,
platform agnostic and may even remove the need for dumping the pixel
in some cases.

Even if the existing harness has the subsequent paint issue, having
the pixels for the repaint test is one of the few cases where they are
useful IMHO (sometimes the exact area to repaint is a judgement call,
see e.g. the differences between the platforms). Also there are
several of techniques to make the pixel dump platform-agnostic which
could minimize the number of baselines.

On top of that, if both your test case and reference hit the same bug,
it nullifies the value of your test - this may be a theoretical issue
though as pointed out in the pixel vs ref-test thread. I didn't
understand from your proposal how you would avoid this risk.

Thanks,
Julien