[webkit-dev] Pixel tests and displaying text

[Dan Bernstein]

On Dec 8, 2010, at 12:13 PM, Peter Kasting wrote:

> If you never write layout tests, you can stop reading.
> 
> Right now few ports run pixel tests (a shame).  One reason is because we frequently need different reference images for each port, and creating hundreds or thousands of these is a hassle.  Maintenance burden also increases.
> 
> We could greatly decrease the number of these baselines by following a simple rule: don't display text unless you need to.  For example, let's say I have a test that is supposed to display a green square.  If the test output is:
> 
> A green square means this test passes.  If there is any red below, fail.
> [green square here]
> 
> ...then for each platform that has different font rendering, whether that be subpixel AA differences, font size differences, or anything else, we'll need new baselines.  By contrast, if that explanation was in an HTML comment inside the test code, and the output was a simple green square, one baseline would serve for all ports.
> 
> I think this doesn't really hinder tracking down test failures, for a couple of reasons:
> * Most tests are "green = pass, red = failure" by convention, so frequently a pixel result that differs from the baseline is easily classifiable at a quick glance.
> * When this isn't the case, one of the first steps in understanding the test output is to read the test, at which point the HTML comment will explain things.
> 
> Obviously, some tests need to display text, but this seems to me to be a good rule of thumb.

Tests that involve text can still have cross-platform results if they use the Ahem font. See for example <http://trac.webkit.org/browser/trunk/LayoutTests/fast/css/first-letter-set-text.html?rev=55196>.

> 
> Am I missing anything?  If not, is anyone interested in helping to make this change on existing tests where possible, to trim the number of baselines in the tree and make it easier for new ports to bring up pixel tests?
> 
> PK
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20101208/c7d51e96/attachment.html>