[webkit-dev] testharness Wiki page added

[Maciej Stachowiak]

On May 31, 2012, at 5:51 PM, Jacob Goldstein <jacobg at adobe.com> wrote:

> I haven't found that to be the case for the tests I have written for each suite, the output from testharness can be as simple as "PASS" or "FAIL", or include additional debug information defined by the test author.  That being said, my experience is likely more limited than yours.  Peter Linss, who wrote and maintains the W3C testharness, would likely be open to suggestions for improvement if there is anything specific that we want to change.   

The source is more verbose. The output is uglier and less clear, at least by default. "PASS" and "FAIL" is not as good as what our test driver does by default, because you can't as easily tell what passed or failed or why. Our own harness tells you the expression the test evaluated that failed, and what the expected result was, so it's much quicker to debug a failure.

I have given my feedback to James Graham before, including showing him how our test harness works. At the time, he was not open to making changes, in part because of details of Opera's internal testing setup and in part because "eval is evil". I really would rather not reduce all our DOM tests to the Opera-driven level of source legibility and output quality.

Let me give you an example. This zip file contains an actual w3c test case, and a webkit-style conversion of the same test:



I have deliberately introduced the same failure into both. Which one do you think would be easier to debug?

Regards,
Maciej

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120531/8742cb6e/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test-example.zip
Type: application/zip
Size: 31005 bytes
Desc: not available
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120531/8742cb6e/attachment-0001.zip>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120531/8742cb6e/attachment-0003.html>