[webkit-dev] Test conversion to use W3C testharness.js

[Adam Barth]

Ah, I see now that the word "Pass" is there at the beginning of the
line.  I missed it because it ran together with the next couple words.
 Having spaces between the words would address my concern.

Thanks!
Adam


2012/3/9 Jacob Goldstein <jacobg at adobe.com>:
> Is your concern that the Pass is not in all caps?
>
> The current output looks like this:
>
> Result  Test Name       Message
> Pass    testParse: Assigned none to property, expected return = none
>
>
> The pass at the beginning of the line is the test result.  Had the test
> failed, this would read "Fail".
>
> The next portion of the line, in this case "testParse: Assigned none to
> property, expected return = none", is the description string specified in
> the call to assert_equals.  The description can be any string that the
> user specifies.
>
> The spacing of the results when they are dumped as text obscures the
> difference between the test result and description - this is something I
> can address in a number of ways depending on everyone's preference.  I'll
> investigate further and come up with some suggested fixes.
>
>
>
>
>
> On 3/9/12 4:11 PM, "Adam Barth" <abarth at webkit.org> wrote:
>
>>Can we use some CSS tricks like :before to put the word PASS at the
>>beginning of the line for each subtest?
>>
>>Adam
>>
>>
>>On Fri, Mar 9, 2012 at 3:51 PM, Jacob Goldstein <jacobg at adobe.com> wrote:
>>> I replied to your comment in the bug, but will also copy below
>>> -----
>>>
>>> I understand your concern.  Some of this is user defined, and the output
>>> could also be improved via customizations to the testharnessreport.js
>>>file.
>>>
>>> The message you see after "Pass", for example, "testParse: Assigned none
>>> to property, expected return = none", is the user defined string from
>>>the
>>> description argument of the assert_equals method.   assert_equals looks
>>> like this:
>>>
>>> assert_equals(actual, expected, description)
>>>
>>> All of the methods in testharness.js include a description argument that
>>> is output after the test result.  In the W3C test suite, the
>>> testharness.css file formats the output into a slightly more readable
>>> output.  Since we are dumping as text, that isn't possible, but other
>>> customizations in testharnessreport.js are.  I will look into this
>>>further.
>>>
>>>
>>>
>>>
>>>
>>> On 3/9/12 3:21 PM, "Adam Barth" <abarth at webkit.org> wrote:
>>>
>>>>I looked at the patch you uploaded, but it wasn't clear from the text
>>>>dump whether the subtests passed or failed.  Maybe testharness.js uses
>>>>a table and/or colors to present that information?  It's important
>>>>that we can easily determine which subtests pass or fail from a text
>>>>dump.
>>>>
>>>>Adam
>>>>
>>>>
>>>>On Fri, Mar 9, 2012 at 3:19 PM, Jacob Goldstein <jacobg at adobe.com>
>>>>wrote:
>>>>> LayoutTests/resources is fine with me ­ that was the location I
>>>>>considered
>>>>> using originally and only moved them to LayoutTests/fast/js/resources
>>>>> because that is where js-test-pre and ­post are.
>>>>>
>>>>> I'll upload a new patch with the files in LayoutTests/resources.
>>>>>
>>>>>
>>>>>
>>>>> From: Ryosuke Niwa <rniwa at webkit.org>
>>>>> Date: Fri, 9 Mar 2012 14:37:18 -0800
>>>>> To: Jacob Goldstein <jacobg at adobe.com>
>>>>> Cc: "webkit-dev at lists.webkit.org" <webkit-dev at lists.webkit.org>
>>>>> Subject: Re: [webkit-dev] Test conversion to use W3C testharness.js
>>>>>
>>>>> On Fri, Mar 9, 2012 at 2:28 PM, Jacob Goldstein <jacobg at adobe.com>
>>>>>wrote:
>>>>>>
>>>>>> I recently uploaded a patch
>>>>>> to https://bugs.webkit.org/show_bug.cgi?id=80709 which converted an
>>>>>>existing
>>>>>> JavaScript regions parsing test to use the W3C testharness.js in
>>>>>>place
>>>>>>of
>>>>>> js-test-pre.js/js-test-post.js.  This patch also places
>>>>>>testharness.js
>>>>>>and a
>>>>>> WebKit-specific testharnessreport.js file in
>>>>>>LayoutTests/fast/js/resources.
>>>>>
>>>>>
>>>>> Can we put them in LayoutTests/resources instead? I always find it
>>>>>hard
>>>>>to
>>>>> remember the path fast/js/resources.
>>>>>
>>>>>> In cases where tests need to be written for both the WebKit and W3C
>>>>>> testing suites, having the ability to use testharness.js with WebKit
>>>>>>tests
>>>>>> would mean that the test file only needs to be written once, and yet
>>>>>>can
>>>>>> still rely on the functionality from both test harnesses.   As it
>>>>>>stands
>>>>>> now, if someone needs to write a test for both suites, they either
>>>>>>have to
>>>>>> implement all functionality from scratch, or write one version of the
>>>>>>test
>>>>>> to use js-test-pre.js and another to use testharness.js.  The
>>>>>>inclusion of
>>>>>> testharness.js in the WebKit repository alleviates the need for this
>>>>>> duplication of effort.  The testharnessreport.js file was intended
>>>>>>for
>>>>>> customization of the capabilities provided by testharness.js, I've
>>>>>>added a
>>>>>> call to layoutTestController.dumpAsText() to this file to allow it to
>>>>>> function as a WebKit JavaScript test.
>>>>>
>>>>>
>>>>> I support the effort to make layout tests more compatible with W3C
>>>>>tests.
>>>>>
>>>>> Is the plan to use testharness.js for all new tests? Or only tests
>>>>>that
>>>>>we
>>>>> intend to contribute back to W3C?
>>>>>
>>>>>> Another concern is that changes to testharness.js in the future that
>>>>>>break
>>>>>> backward compatibility could then break WebKit tests.  This is an
>>>>>>issue I
>>>>>> plan to discuss with W3C members to determine if backward
>>>>>>compatibility can
>>>>>> be ensured.
>>>>>
>>>>>
>>>>> There is no such a guarantee at the moment? That concerns me. On other
>>>>>hand,
>>>>> we wouldn't be importing ToT version of testharness.js so if such an
>>>>> incompatibility is introduced, we can migrate our tests on time as
>>>>>well.
>>>>>
>>>>> - Ryosuke
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> webkit-dev mailing list
>>>>> webkit-dev at lists.webkit.org
>>>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>>>>
>>>
>