[webkit-dev] Test conversion to use W3C testharness.js

Jacob Goldstein jacobg at adobe.com
Fri Mar 9 17:17:35 PST 2012 

Got it. I'll add more space after the Pass and see what other formatting makes sense and get back to you next week.



On Mar 9, 2012, at 5:15 PM, "Adam Barth" <abarth at webkit.org> wrote:

> Ah, I see now that the word "Pass" is there at the beginning of the
> line.  I missed it because it ran together with the next couple words.
> Having spaces between the words would address my concern.
> 
> Thanks!
> Adam
> 
> 
> 2012/3/9 Jacob Goldstein <jacobg at adobe.com>:
>> Is your concern that the Pass is not in all caps?
>> 
>> The current output looks like this:
>> 
>> Result  Test Name       Message
>> Pass    testParse: Assigned none to property, expected return = none
>> 
>> 
>> The pass at the beginning of the line is the test result.  Had the test
>> failed, this would read "Fail".
>> 
>> The next portion of the line, in this case "testParse: Assigned none to
>> property, expected return = none", is the description string specified in
>> the call to assert_equals.  The description can be any string that the
>> user specifies.
>> 
>> The spacing of the results when they are dumped as text obscures the
>> difference between the test result and description - this is something I
>> can address in a number of ways depending on everyone's preference.  I'll
>> investigate further and come up with some suggested fixes.
>> 
>> 
>> 
>> 
>> 
>> On 3/9/12 4:11 PM, "Adam Barth" <abarth at webkit.org> wrote:
>> 
>>> Can we use some CSS tricks like :before to put the word PASS at the
>>> beginning of the line for each subtest?
>>> 
>>> Adam
>>> 
>>> 
>>> On Fri, Mar 9, 2012 at 3:51 PM, Jacob Goldstein <jacobg at adobe.com> wrote:
>>>> I replied to your comment in the bug, but will also copy below
>>>> -----
>>>> 
>>>> I understand your concern.  Some of this is user defined, and the output
>>>> could also be improved via customizations to the testharnessreport.js
>>>> file.
>>>> 
>>>> The message you see after "Pass", for example, "testParse: Assigned none
>>>> to property, expected return = none", is the user defined string from
>>>> the
>>>> description argument of the assert_equals method.   assert_equals looks
>>>> like this:
>>>> 
>>>> assert_equals(actual, expected, description)
>>>> 
>>>> All of the methods in testharness.js include a description argument that
>>>> is output after the test result.  In the W3C test suite, the
>>>> testharness.css file formats the output into a slightly more readable
>>>> output.  Since we are dumping as text, that isn't possible, but other
>>>> customizations in testharnessreport.js are.  I will look into this
>>>> further.
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On 3/9/12 3:21 PM, "Adam Barth" <abarth at webkit.org> wrote:
>>>> 
>>>>> I looked at the patch you uploaded, but it wasn't clear from the text
>>>>> dump whether the subtests passed or failed.  Maybe testharness.js uses
>>>>> a table and/or colors to present that information?  It's important
>>>>> that we can easily determine which subtests pass or fail from a text
>>>>> dump.
>>>>> 
>>>>> Adam
>>>>> 
>>>>> 
>>>>> On Fri, Mar 9, 2012 at 3:19 PM, Jacob Goldstein <jacobg at adobe.com>
>>>>> wrote:
>>>>>> LayoutTests/resources is fine with me ­ that was the location I
>>>>>> considered
>>>>>> using originally and only moved them to LayoutTests/fast/js/resources
>>>>>> because that is where js-test-pre and ­post are.
>>>>>> 
>>>>>> I'll upload a new patch with the files in LayoutTests/resources.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> From: Ryosuke Niwa <rniwa at webkit.org>
>>>>>> Date: Fri, 9 Mar 2012 14:37:18 -0800
>>>>>> To: Jacob Goldstein <jacobg at adobe.com>
>>>>>> Cc: "webkit-dev at lists.webkit.org" <webkit-dev at lists.webkit.org>
>>>>>> Subject: Re: [webkit-dev] Test conversion to use W3C testharness.js
>>>>>> 
>>>>>> On Fri, Mar 9, 2012 at 2:28 PM, Jacob Goldstein <jacobg at adobe.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>> I recently uploaded a patch
>>>>>>> to https://bugs.webkit.org/show_bug.cgi?id=80709 which converted an
>>>>>>> existing
>>>>>>> JavaScript regions parsing test to use the W3C testharness.js in
>>>>>>> place
>>>>>>> of
>>>>>>> js-test-pre.js/js-test-post.js.  This patch also places
>>>>>>> testharness.js
>>>>>>> and a
>>>>>>> WebKit-specific testharnessreport.js file in
>>>>>>> LayoutTests/fast/js/resources.
>>>>>> 
>>>>>> 
>>>>>> Can we put them in LayoutTests/resources instead? I always find it
>>>>>> hard
>>>>>> to
>>>>>> remember the path fast/js/resources.
>>>>>> 
>>>>>>> In cases where tests need to be written for both the WebKit and W3C
>>>>>>> testing suites, having the ability to use testharness.js with WebKit
>>>>>>> tests
>>>>>>> would mean that the test file only needs to be written once, and yet
>>>>>>> can
>>>>>>> still rely on the functionality from both test harnesses.   As it
>>>>>>> stands
>>>>>>> now, if someone needs to write a test for both suites, they either
>>>>>>> have to
>>>>>>> implement all functionality from scratch, or write one version of the
>>>>>>> test
>>>>>>> to use js-test-pre.js and another to use testharness.js.  The
>>>>>>> inclusion of
>>>>>>> testharness.js in the WebKit repository alleviates the need for this
>>>>>>> duplication of effort.  The testharnessreport.js file was intended
>>>>>>> for
>>>>>>> customization of the capabilities provided by testharness.js, I've
>>>>>>> added a
>>>>>>> call to layoutTestController.dumpAsText() to this file to allow it to
>>>>>>> function as a WebKit JavaScript test.
>>>>>> 
>>>>>> 
>>>>>> I support the effort to make layout tests more compatible with W3C
>>>>>> tests.
>>>>>> 
>>>>>> Is the plan to use testharness.js for all new tests? Or only tests
>>>>>> that
>>>>>> we
>>>>>> intend to contribute back to W3C?
>>>>>> 
>>>>>>> Another concern is that changes to testharness.js in the future that
>>>>>>> break
>>>>>>> backward compatibility could then break WebKit tests.  This is an
>>>>>>> issue I
>>>>>>> plan to discuss with W3C members to determine if backward
>>>>>>> compatibility can
>>>>>>> be ensured.
>>>>>> 
>>>>>> 
>>>>>> There is no such a guarantee at the moment? That concerns me. On other
>>>>>> hand,
>>>>>> we wouldn't be importing ToT version of testharness.js so if such an
>>>>>> incompatibility is introduced, we can migrate our tests on time as
>>>>>> well.
>>>>>> 
>>>>>> - Ryosuke
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> webkit-dev mailing list
>>>>>> webkit-dev at lists.webkit.org
>>>>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>>>>> 
>>>> 
>>

More information about the webkit-dev mailing list