[webkit-dev] Test conversion to use W3C testharness.js

[Jacob Goldstein]

I replied to your comment in the bug, but will also copy below
-----

I understand your concern.  Some of this is user defined, and the output
could also be improved via customizations to the testharnessreport.js file.

The message you see after "Pass", for example, "testParse: Assigned none
to property, expected return = none", is the user defined string from the
description argument of the assert_equals method.   assert_equals looks
like this:

assert_equals(actual, expected, description)

All of the methods in testharness.js include a description argument that
is output after the test result.  In the W3C test suite, the
testharness.css file formats the output into a slightly more readable
output.  Since we are dumping as text, that isn't possible, but other
customizations in testharnessreport.js are.  I will look into this further.





On 3/9/12 3:21 PM, "Adam Barth" <abarth at webkit.org> wrote:

>I looked at the patch you uploaded, but it wasn't clear from the text
>dump whether the subtests passed or failed.  Maybe testharness.js uses
>a table and/or colors to present that information?  It's important
>that we can easily determine which subtests pass or fail from a text
>dump.
>
>Adam
>
>
>On Fri, Mar 9, 2012 at 3:19 PM, Jacob Goldstein <jacobg at adobe.com> wrote:
>> LayoutTests/resources is fine with me ­ that was the location I
>>considered
>> using originally and only moved them to LayoutTests/fast/js/resources
>> because that is where js-test-pre and ­post are.
>>
>> I'll upload a new patch with the files in LayoutTests/resources.
>>
>>
>>
>> From: Ryosuke Niwa <rniwa at webkit.org>
>> Date: Fri, 9 Mar 2012 14:37:18 -0800
>> To: Jacob Goldstein <jacobg at adobe.com>
>> Cc: "webkit-dev at lists.webkit.org" <webkit-dev at lists.webkit.org>
>> Subject: Re: [webkit-dev] Test conversion to use W3C testharness.js
>>
>> On Fri, Mar 9, 2012 at 2:28 PM, Jacob Goldstein <jacobg at adobe.com>
>>wrote:
>>>
>>> I recently uploaded a patch
>>> to https://bugs.webkit.org/show_bug.cgi?id=80709 which converted an
>>>existing
>>> JavaScript regions parsing test to use the W3C testharness.js in place
>>>of
>>> js-test-pre.js/js-test-post.js.  This patch also places testharness.js
>>>and a
>>> WebKit-specific testharnessreport.js file in
>>>LayoutTests/fast/js/resources.
>>
>>
>> Can we put them in LayoutTests/resources instead? I always find it hard
>>to
>> remember the path fast/js/resources.
>>
>>> In cases where tests need to be written for both the WebKit and W3C
>>> testing suites, having the ability to use testharness.js with WebKit
>>>tests
>>> would mean that the test file only needs to be written once, and yet
>>>can
>>> still rely on the functionality from both test harnesses.   As it
>>>stands
>>> now, if someone needs to write a test for both suites, they either
>>>have to
>>> implement all functionality from scratch, or write one version of the
>>>test
>>> to use js-test-pre.js and another to use testharness.js.  The
>>>inclusion of
>>> testharness.js in the WebKit repository alleviates the need for this
>>> duplication of effort.  The testharnessreport.js file was intended for
>>> customization of the capabilities provided by testharness.js, I've
>>>added a
>>> call to layoutTestController.dumpAsText() to this file to allow it to
>>> function as a WebKit JavaScript test.
>>
>>
>> I support the effort to make layout tests more compatible with W3C
>>tests.
>>
>> Is the plan to use testharness.js for all new tests? Or only tests that
>>we
>> intend to contribute back to W3C?
>>
>>> Another concern is that changes to testharness.js in the future that
>>>break
>>> backward compatibility could then break WebKit tests.  This is an
>>>issue I
>>> plan to discuss with W3C members to determine if backward
>>>compatibility can
>>> be ensured.
>>
>>
>> There is no such a guarantee at the moment? That concerns me. On other
>>hand,
>> we wouldn't be importing ToT version of testharness.js so if such an
>> incompatibility is introduced, we can migrate our tests on time as well.
>>
>> - Ryosuke
>>
>>
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>