[webkit-dev] new-run-webkit-tests and Mac pixel tests

Dirk Pranke dpranke at chromium.org
Fri Mar 26 18:32:08 PDT 2010

Hi all,

I have just landed a patch that updates a bunch of incorrect image
checksums, and adds all of the currently failing pixel tests to the
"test_expectations.txt" file in LayoutTests/platform/mac.

If you take a look at that file you'll see that they are marked with a
BUG36620 string and the string "IMAGE" to indicate that they are
expected to pass as text-only tests but fail image diffs.

If you use new-run-webkit-tests, pixel tests are now enabled by
default on the Mac, and you should get a clean test run indicating
that all tests run as expected.

Hopefully this is a first step to getting pixel tests passing all the
time (which I realize may be a controversial goal).

Apart from the fact that the patch fixed a bunch of incorrect
checksums, the standard run-webkit-tests should be unaffected.

-- Dirk

[1] If you disable pixel tests (using --no-pixel-tests), then you will
get a bunch of warnings that tests expected to fail with pixel
failures are passing (because the test infrastructure isn't smart
enough yet to realize that we're not doing pixel tests). See

[2] Also, if you run on a machine with more than 4 CPUs, you may find
new-run-webkit-tests hanging quite frequently. This is a known bug [3]
that we are trying to hunt down. I have a patch out to change the
default in the meantime [4]. To work around this, simply use
--num-test-shells=4 (or less).

[3] https://bugs.webkit.org/show_bug.cgi?id=36622

[4] https://bugs.webkit.org/show_bug.cgi?id=36687

More information about the webkit-dev mailing list