[webkit-dev] Using ref tests for repaint bugs

[Andrei Bucur]

On Fri, May 25, 2012 at 4:05 PM, Julien Chaffraix <
julien.chaffraix at gmail.com> wrote:
>
> I second Simon's fragility argument. His solution is more viable,
> platform agnostic and may even remove the need for dumping the pixel
> in some cases.
>
> Even if the existing harness has the subsequent paint issue, having
> the pixels for the repaint test is one of the few cases where they are
> useful IMHO (sometimes the exact area to repaint is a judgement call,
> see e.g. the differences between the platforms). Also there are
> several of techniques to make the pixel dump platform-agnostic which
> could minimize the number of baselines.
>
> On top of that, if both your test case and reference hit the same bug,
> it nullifies the value of your test - this may be a theoretical issue
> though as pointed out in the pixel vs ref-test thread. I didn't
> understand from your proposal how you would avoid this risk.
>
> Thanks,
> Julien
>

In some cases I don't see how the reference and test files can hit the same
bug that you are testing for if the test file is correctly designed.

Imagine a simple scenario with a background repaint bug. The test document
has an element that's 50px tall with a background color of red. After
making the element 25px tall, the surface is not correctly cleared and the
red background remains 50px tall. If the reference document contains the
element directly with 25px height, it's pretty impossible WebKit will ever
draw it with a 50px tall background, thus the ref-test will detect if that
specific repaint bug exists.

I never said ref-tests can replace all the pixel tests, but for bugs like
the one I've mentioned above they seem to do quite a good job.

Simon's idea sounds very good as well, but it has the disadvantage of a
maintenance cost greater than the one for a ref-test.

I suppose the best thing is to have both of them implemented and based on
the specificity of the bug tested, use one or another. For straightforward
repaint bugs, ref tests sound good. For more complex bugs, comparing
repaint rectangles seems better. I'll start prototyping both of them as it
doesn't look extremely difficult to extract the repaint rectangle
information. Afterwards I'll try to see how both approaches can be applied
to the current repaint testing battery.

Thanks,
Andrei.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120528/b844b28e/attachment.html>