[webkit-dev] Test conversion to use W3C testharness.js

[Jacob Goldstein]

Is your concern that the Pass is not in all caps?

The current output looks like this:

Result	Test Name	Message
Pass	testParse: Assigned none to property, expected return = none


The pass at the beginning of the line is the test result.  Had the test
failed, this would read "Fail".

The next portion of the line, in this case "testParse: Assigned none to
property, expected return = none", is the description string specified in
the call to assert_equals.  The description can be any string that the
user specifies.

The spacing of the results when they are dumped as text obscures the
difference between the test result and description - this is something I
can address in a number of ways depending on everyone's preference.  I'll
investigate further and come up with some suggested fixes.





On 3/9/12 4:11 PM, "Adam Barth" <abarth at webkit.org> wrote:

>Can we use some CSS tricks like :before to put the word PASS at the
>beginning of the line for each subtest?
>
>Adam
>
>
>On Fri, Mar 9, 2012 at 3:51 PM, Jacob Goldstein <jacobg at adobe.com> wrote:
>> I replied to your comment in the bug, but will also copy below
>> -----
>>
>> I understand your concern.  Some of this is user defined, and the output
>> could also be improved via customizations to the testharnessreport.js
>>file.
>>
>> The message you see after "Pass", for example, "testParse: Assigned none
>> to property, expected return = none", is the user defined string from
>>the
>> description argument of the assert_equals method.   assert_equals looks
>> like this:
>>
>> assert_equals(actual, expected, description)
>>
>> All of the methods in testharness.js include a description argument that
>> is output after the test result.  In the W3C test suite, the
>> testharness.css file formats the output into a slightly more readable
>> output.  Since we are dumping as text, that isn't possible, but other
>> customizations in testharnessreport.js are.  I will look into this
>>further.
>>
>>
>>
>>
>>
>> On 3/9/12 3:21 PM, "Adam Barth" <abarth at webkit.org> wrote:
>>
>>>I looked at the patch you uploaded, but it wasn't clear from the text
>>>dump whether the subtests passed or failed.  Maybe testharness.js uses
>>>a table and/or colors to present that information?  It's important
>>>that we can easily determine which subtests pass or fail from a text
>>>dump.
>>>
>>>Adam
>>>
>>>
>>>On Fri, Mar 9, 2012 at 3:19 PM, Jacob Goldstein <jacobg at adobe.com>
>>>wrote:
>>>> LayoutTests/resources is fine with me ­ that was the location I
>>>>considered
>>>> using originally and only moved them to LayoutTests/fast/js/resources
>>>> because that is where js-test-pre and ­post are.
>>>>
>>>> I'll upload a new patch with the files in LayoutTests/resources.
>>>>
>>>>
>>>>
>>>> From: Ryosuke Niwa <rniwa at webkit.org>
>>>> Date: Fri, 9 Mar 2012 14:37:18 -0800
>>>> To: Jacob Goldstein <jacobg at adobe.com>
>>>> Cc: "webkit-dev at lists.webkit.org" <webkit-dev at lists.webkit.org>
>>>> Subject: Re: [webkit-dev] Test conversion to use W3C testharness.js
>>>>
>>>> On Fri, Mar 9, 2012 at 2:28 PM, Jacob Goldstein <jacobg at adobe.com>
>>>>wrote:
>>>>>
>>>>> I recently uploaded a patch
>>>>> to https://bugs.webkit.org/show_bug.cgi?id=80709 which converted an
>>>>>existing
>>>>> JavaScript regions parsing test to use the W3C testharness.js in
>>>>>place
>>>>>of
>>>>> js-test-pre.js/js-test-post.js.  This patch also places
>>>>>testharness.js
>>>>>and a
>>>>> WebKit-specific testharnessreport.js file in
>>>>>LayoutTests/fast/js/resources.
>>>>
>>>>
>>>> Can we put them in LayoutTests/resources instead? I always find it
>>>>hard
>>>>to
>>>> remember the path fast/js/resources.
>>>>
>>>>> In cases where tests need to be written for both the WebKit and W3C
>>>>> testing suites, having the ability to use testharness.js with WebKit
>>>>>tests
>>>>> would mean that the test file only needs to be written once, and yet
>>>>>can
>>>>> still rely on the functionality from both test harnesses.   As it
>>>>>stands
>>>>> now, if someone needs to write a test for both suites, they either
>>>>>have to
>>>>> implement all functionality from scratch, or write one version of the
>>>>>test
>>>>> to use js-test-pre.js and another to use testharness.js.  The
>>>>>inclusion of
>>>>> testharness.js in the WebKit repository alleviates the need for this
>>>>> duplication of effort.  The testharnessreport.js file was intended
>>>>>for
>>>>> customization of the capabilities provided by testharness.js, I've
>>>>>added a
>>>>> call to layoutTestController.dumpAsText() to this file to allow it to
>>>>> function as a WebKit JavaScript test.
>>>>
>>>>
>>>> I support the effort to make layout tests more compatible with W3C
>>>>tests.
>>>>
>>>> Is the plan to use testharness.js for all new tests? Or only tests
>>>>that
>>>>we
>>>> intend to contribute back to W3C?
>>>>
>>>>> Another concern is that changes to testharness.js in the future that
>>>>>break
>>>>> backward compatibility could then break WebKit tests.  This is an
>>>>>issue I
>>>>> plan to discuss with W3C members to determine if backward
>>>>>compatibility can
>>>>> be ensured.
>>>>
>>>>
>>>> There is no such a guarantee at the moment? That concerns me. On other
>>>>hand,
>>>> we wouldn't be importing ToT version of testharness.js so if such an
>>>> incompatibility is introduced, we can migrate our tests on time as
>>>>well.
>>>>
>>>> - Ryosuke
>>>>
>>>>
>>>> _______________________________________________
>>>> webkit-dev mailing list
>>>> webkit-dev at lists.webkit.org
>>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>>>
>>