[webkit-dev] Adding New Layout Tests
David D. Kilzer
ddkilzer at kilzer.net
Sat Jun 2 23:28:59 PDT 2007
Reem Yazigi <reemyazigi at hotmail.com> wrote:
> Thanks for your helpful answers. I still have some more questions for you
> 1- Dave said "Image test is required to capture expected results that a text
> based test can't". Can you please provide me with a path/name of one of these
$ ls LayoutTests/fast/images/*.html
> 2- I actually need to know which of the layout test cases I'm adding requires
> Image test, so I can generate the PNGs and Checksums for these specific test
> cases (through the -p option of the "run-webkit-tests" script)- Or do you
> think that it's better to generate the PNGs and checksums for all the test
> I'm planning to add?
If you can write a test (that breaks before fixing the code) by writing a
text-only test case (see "layoutTestController.dumpAsText() in
LayoutTests/fast/css/computed-style.html), do that. Image tests take a lot
longer to run and should be avoided unless the test requires it.
Otherwise, new test cases should have both text and pixel results if they're
not text-only tests. (If you're porting to a new platform, I'd get the
text-only tests working first, then worry about the image tests later.)
> 3- How is the PNGs comparison implemented for regression testing? (in the
> text based tests it's a string comparison how about the image based tests?)
Checksums of the two images are compared. I take it you haven't looked through
the source of the run-webkit-tests script yet?
> 4- What are the Checksum files used for?
> 5- So, please correct me if there's anything missing in the following steps:
> In order to test a WebKit based port other than Safari and Qt with WebKit's
> Layout tests I need to:
> a- update the "run-webkit-tests" Perl script.
> b- port the "DumpRenderTree" app.
> c- regenerate the expected results for each of the layout tests
> (knowing that the render tree dumps are different for each
Test results for a new platform should go under the LayoutTestResults directory
in a new "platform" directory. Please make sure that the results are
repeatable, e.g., running the tests again after generating the results (both
individually and as a group) don't generate any differences unless the WebKit
code changes as well.
More information about the webkit-dev