[webkit-dev] Pixel tests and displaying text

[Simon Fraser]

On Dec 8, 2010, at 12:13 PM, Peter Kasting wrote:

> If you never write layout tests, you can stop reading.
> 
> Right now few ports run pixel tests (a shame).  One reason is because we frequently need different reference images for each port, and creating hundreds or thousands of these is a hassle.  Maintenance burden also increases.
> 
> We could greatly decrease the number of these baselines by following a simple rule: don't display text unless you need to.  For example, let's say I have a test that is supposed to display a green square.  If the test output is:
> 
> A green square means this test passes.  If there is any red below, fail.
> [green square here]
> 
> ...then for each platform that has different font rendering, whether that be subpixel AA differences, font size differences, or anything else, we'll need new baselines.  By contrast, if that explanation was in an HTML comment inside the test code, and the output was a simple green square, one baseline would serve for all ports.
> 
> I think this doesn't really hinder tracking down test failures, for a couple of reasons:
> * Most tests are "green = pass, red = failure" by convention, so frequently a pixel result that differs from the baseline is easily classifiable at a quick glance.
> * When this isn't the case, one of the first steps in understanding the test output is to read the test, at which point the HTML comment will explain things.
> 
> Obviously, some tests need to display text, but this seems to me to be a good rule of thumb.
> 
> Am I missing anything?  If not, is anyone interested in helping to make this change on existing tests where possible, to trim the number of baselines in the tree and make it easier for new ports to bring up pixel tests?

I totally agree, and have been avoiding test in pixel tests as much as possible recently.

One useful strategy is to add explanatory notes to the test as HTML comments, so the purpose of the test is easily determined by someone looking at the source.

Simon


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20101208/0712763e/attachment.html>