[webkit-dev] Test conversion to use W3C testharness.js

Adam Barth abarth at webkit.org
Fri Mar 9 17:14:43 PST 2012

Ah, I see now that the word "Pass" is there at the beginning of the
line.  I missed it because it ran together with the next couple words.
 Having spaces between the words would address my concern.


2012/3/9 Jacob Goldstein <jacobg at adobe.com>:
> Is your concern that the Pass is not in all caps?
> The current output looks like this:
> Result  Test Name       Message
> Pass    testParse: Assigned none to property, expected return = none
> The pass at the beginning of the line is the test result.  Had the test
> failed, this would read "Fail".
> The next portion of the line, in this case "testParse: Assigned none to
> property, expected return = none", is the description string specified in
> the call to assert_equals.  The description can be any string that the
> user specifies.
> The spacing of the results when they are dumped as text obscures the
> difference between the test result and description - this is something I
> can address in a number of ways depending on everyone's preference.  I'll
> investigate further and come up with some suggested fixes.
> On 3/9/12 4:11 PM, "Adam Barth" <abarth at webkit.org> wrote:
>>Can we use some CSS tricks like :before to put the word PASS at the
>>beginning of the line for each subtest?
>>On Fri, Mar 9, 2012 at 3:51 PM, Jacob Goldstein <jacobg at adobe.com> wrote:
>>> I replied to your comment in the bug, but will also copy below
>>> -----
>>> I understand your concern.  Some of this is user defined, and the output
>>> could also be improved via customizations to the testharnessreport.js
>>> The message you see after "Pass", for example, "testParse: Assigned none
>>> to property, expected return = none", is the user defined string from
>>> description argument of the assert_equals method.   assert_equals looks
>>> like this:
>>> assert_equals(actual, expected, description)
>>> All of the methods in testharness.js include a description argument that
>>> is output after the test result.  In the W3C test suite, the
>>> testharness.css file formats the output into a slightly more readable
>>> output.  Since we are dumping as text, that isn't possible, but other
>>> customizations in testharnessreport.js are.  I will look into this
>>> On 3/9/12 3:21 PM, "Adam Barth" <abarth at webkit.org> wrote:
>>>>I looked at the patch you uploaded, but it wasn't clear from the text
>>>>dump whether the subtests passed or failed.  Maybe testharness.js uses
>>>>a table and/or colors to present that information?  It's important
>>>>that we can easily determine which subtests pass or fail from a text
>>>>On Fri, Mar 9, 2012 at 3:19 PM, Jacob Goldstein <jacobg at adobe.com>
>>>>> LayoutTests/resources is fine with me ­ that was the location I
>>>>> using originally and only moved them to LayoutTests/fast/js/resources
>>>>> because that is where js-test-pre and ­post are.
>>>>> I'll upload a new patch with the files in LayoutTests/resources.
>>>>> From: Ryosuke Niwa <rniwa at webkit.org>
>>>>> Date: Fri, 9 Mar 2012 14:37:18 -0800
>>>>> To: Jacob Goldstein <jacobg at adobe.com>
>>>>> Cc: "webkit-dev at lists.webkit.org" <webkit-dev at lists.webkit.org>
>>>>> Subject: Re: [webkit-dev] Test conversion to use W3C testharness.js
>>>>> On Fri, Mar 9, 2012 at 2:28 PM, Jacob Goldstein <jacobg at adobe.com>
>>>>>> I recently uploaded a patch
>>>>>> to https://bugs.webkit.org/show_bug.cgi?id=80709 which converted an
>>>>>> JavaScript regions parsing test to use the W3C testharness.js in
>>>>>> js-test-pre.js/js-test-post.js.  This patch also places
>>>>>>and a
>>>>>> WebKit-specific testharnessreport.js file in
>>>>> Can we put them in LayoutTests/resources instead? I always find it
>>>>> remember the path fast/js/resources.
>>>>>> In cases where tests need to be written for both the WebKit and W3C
>>>>>> testing suites, having the ability to use testharness.js with WebKit
>>>>>> would mean that the test file only needs to be written once, and yet
>>>>>> still rely on the functionality from both test harnesses.   As it
>>>>>> now, if someone needs to write a test for both suites, they either
>>>>>>have to
>>>>>> implement all functionality from scratch, or write one version of the
>>>>>> to use js-test-pre.js and another to use testharness.js.  The
>>>>>>inclusion of
>>>>>> testharness.js in the WebKit repository alleviates the need for this
>>>>>> duplication of effort.  The testharnessreport.js file was intended
>>>>>> customization of the capabilities provided by testharness.js, I've
>>>>>>added a
>>>>>> call to layoutTestController.dumpAsText() to this file to allow it to
>>>>>> function as a WebKit JavaScript test.
>>>>> I support the effort to make layout tests more compatible with W3C
>>>>> Is the plan to use testharness.js for all new tests? Or only tests
>>>>> intend to contribute back to W3C?
>>>>>> Another concern is that changes to testharness.js in the future that
>>>>>> backward compatibility could then break WebKit tests.  This is an
>>>>>>issue I
>>>>>> plan to discuss with W3C members to determine if backward
>>>>>>compatibility can
>>>>>> be ensured.
>>>>> There is no such a guarantee at the moment? That concerns me. On other
>>>>> we wouldn't be importing ToT version of testharness.js so if such an
>>>>> incompatibility is introduced, we can migrate our tests on time as
>>>>> - Ryosuke
>>>>> _______________________________________________
>>>>> webkit-dev mailing list
>>>>> webkit-dev at lists.webkit.org
>>>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

More information about the webkit-dev mailing list