[webkit-gtk] Pixel tests - 'Shall we?'

[Zan Dobersek]

Hi,

I am writing this email to get some input on the matter of pixel tests in
the Gtk port.

Pixel tests are a good way of checking if any rendering has changed when it
shouldn't. It may seem as a pain to either generate the results or keep them
up to date, but with the right approach this should not be a problem. The
benefits are bigger as I think more progressive testing allows for more
progressive development.

I believe the testing capabilities of the Gtk port have come a long way and
it is reasonable to proceed to the next step, this being enabling the pixel
tests. Before making this step some standards and simple guidelines should
be set up. For example:
1. define what versions of (rendering-related) software (cairo, pixman,
pango etc.) should WebKit be built against,
2. define specific protocol of testing (running Xvfb with specific
arguments, any other commands that should be executed) to ensure equal
results (pretty much done),
3. establish an environment that could be used on the buildbots as well as
on local machines and would produce same results wherever used,
4. update testing structures to enable and ensure proper pixel testing,
5. generating results - before putting them in the source tree, they should
be thoroughly checked to ensure they pass on different machines and perhaps
also the buildbots.

I'd like to hear your thoughts about getting pixel tests going - what are
the pitfalls and what else should be discussed?

If I missed anything essential in the above steps please reply with
information on what should also be done.

Regards,
Zan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-gtk/attachments/20110202/0de6ebbb/attachment.html>