[webkit-dev] testharness Wiki page added

[Dirk Schulze]

On May 31, 2012, at 7:55 PM, Maciej Stachowiak wrote:

> 
> On May 31, 2012, at 5:51 PM, Jacob Goldstein <jacobg at adobe.com> wrote:
> 
>> I haven't found that to be the case for the tests I have written for each suite, the output from testharness can be as simple as "PASS" or "FAIL", or include additional debug information defined by the test author.  That being said, my experience is likely more limited than yours.  Peter Linss, who wrote and maintains the W3C testharness, would likely be open to suggestions for improvement if there is anything specific that we want to change.   
> 
> The source is more verbose. The output is uglier and less clear, at least by default. "PASS" and "FAIL" is not as good as what our test driver does by default, because you can't as easily tell what passed or failed or why. Our own harness tells you the expression the test evaluated that failed, and what the expected result was, so it's much quicker to debug a failure.
> 
> I have given my feedback to James Graham before, including showing him how our test harness works. At the time, he was not open to making changes, in part because of details of Opera's internal testing setup and in part because "eval is evil". I really would rather not reduce all our DOM tests to the Opera-driven level of source legibility and output quality.
I think it is time not to discuss it privately anymore. This is the test suite that is/will be used by the CSS WG. Therefore, if there are reasonable suggestions for improvement, we should discuss it on public mailing lists. webkit-dev is a great start, but since it is used by CSS WG and since all contributors, not just opera contributors, need to use it on the CSS WG test suite, it should be discussed on the CSS WG side. The responsible mailing list should be public-css-testsuite.

> 
> Let me give you an example. This zip file contains an actual w3c test case, and a webkit-style conversion of the same test:
> 
> <test-example.zip>
> 
> I have deliberately introduced the same failure into both. Which one do you think would be easier to debug?
I agree, the output of the WebKit harness seems to be more useful and better for debugging. But I used to use the WebKit testing harness a lot. So my impressions are not independent.

Greetings,
Dirk

> 
> Regards,
> Maciej
> 
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev