[webkit-dev] Pixel tests and displaying text

[Eric Seidel]

I think we should just switch the default for text vs. pixel tests.  Make
tests require a <script> layoutTestController.dumpPixels()</script> to dump
pixels.

We could also white-list certain directories.[1]

-eric

1. If we did add white-listing, I'd like to move away from our
hacky-in-run-webkit-tests white-lists to an explict on-disk per-directory
config file or similar.  Understanding why "fast/loader" turns on the loader
callbacks when you're writing a test has driven more than one
would-be-webkit-hacker crazy. :)

On Wed, Dec 8, 2010 at 12:43 PM, Simon Fraser <simon.fraser at apple.com>wrote:

> On Dec 8, 2010, at 12:13 PM, Peter Kasting wrote:
>
> If you never write layout tests, you can stop reading.
>
> Right now few ports run pixel tests (a shame).  One reason is because we
> frequently need different reference images for each port, and creating
> hundreds or thousands of these is a hassle.  Maintenance burden also
> increases.
>
> We could greatly decrease the number of these baselines by following a
> simple rule: don't display text unless you need to.  For example, let's say
> I have a test that is supposed to display a green square.  If the test
> output is:
>
> A green square means this test passes.  If there is any red below, fail.
> [green square here]
>
> ...then for each platform that has different font rendering, whether that
> be subpixel AA differences, font size differences, or anything else, we'll
> need new baselines.  By contrast, if that explanation was in an HTML comment
> inside the test code, and the output was a simple green square, one baseline
> would serve for all ports.
>
> I think this doesn't really hinder tracking down test failures, for a
> couple of reasons:
> * Most tests are "green = pass, red = failure" by convention, so frequently
> a pixel result that differs from the baseline is easily classifiable at a
> quick glance.
> * When this isn't the case, one of the first steps in understanding the
> test output is to read the test, at which point the HTML comment will
> explain things.
>
> Obviously, some tests need to display text, but this seems to me to be a
> good rule of thumb.
>
> Am I missing anything?  If not, is anyone interested in helping to make
> this change on existing tests where possible, to trim the number of
> baselines in the tree and make it easier for new ports to bring up pixel
> tests?
>
>
> I totally agree, and have been avoiding test in pixel tests as much as
> possible recently.
>
> One useful strategy is to add explanatory notes to the test as HTML
> comments, so the purpose of the test is easily determined by someone looking
> at the source.
>
> Simon
>
>
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20101208/1698186f/attachment.html>