[webkit-dev] Pixel tests and displaying text

[Peter Kasting]

If you never write layout tests, you can stop reading.

Right now few ports run pixel tests (a shame).  One reason is because we
frequently need different reference images for each port, and creating
hundreds or thousands of these is a hassle.  Maintenance burden also
increases.

We could greatly decrease the number of these baselines by following a
simple rule: don't display text unless you need to.  For example, let's say
I have a test that is supposed to display a green square.  If the test
output is:

A green square means this test passes.  If there is any red below, fail.
[green square here]

...then for each platform that has different font rendering, whether that be
subpixel AA differences, font size differences, or anything else, we'll need
new baselines.  By contrast, if that explanation was in an HTML comment
inside the test code, and the output was a simple green square, one baseline
would serve for all ports.

I think this doesn't really hinder tracking down test failures, for a couple
of reasons:
* Most tests are "green = pass, red = failure" by convention, so frequently
a pixel result that differs from the baseline is easily classifiable at a
quick glance.
* When this isn't the case, one of the first steps in understanding the test
output is to read the test, at which point the HTML comment will explain
things.

Obviously, some tests need to display text, but this seems to me to be a
good rule of thumb.

Am I missing anything?  If not, is anyone interested in helping to make this
change on existing tests where possible, to trim the number of baselines in
the tree and make it easier for new ports to bring up pixel tests?

PK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20101208/4e7978b0/attachment.html>