[webkit-dev] Supporting w3c ref tests and changing our convention

[Ryosuke Niwa]

On Fri, Nov 4, 2011 at 7:07 PM, Ojan Vafai <ojan at chromium.org> wrote:

> On Fri, Nov 4, 2011 at 7:03 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:
>>
>> I am, but I'm particularly concerned about W3C tests. It'll be nice if we
>> had exactly one way of running / writing ref tests. I think we can easily
>> automate the process of generating manifest files.
>>
>> The work flow will be as follows:
>>
>>    1. Write new ref tests using link element
>>    2. Run some tool (maybe we can teach run-webkit-tests to do it
>>    automatically)
>>    3. Upload patch with auto-regenerated manifest file
>>
>> It's mainly step 2 that I have a problem with, although I also don't like
> that the test is not self-contained.
>

Generating manifest file when we add a test is much more efficient than
generating it every time we run tests because we tend to do the latter much
more often than the former.

 If we're adding new tests that we intend to contribute back to W3C, then
>>>> those tests should live outside of this directory. The preferred
>>>> approach, however, is to add tests to W3C repo first, then import them back
>>>> to WebKit.
>>>>
>>>
>>> I don't think this is realistic. For example, for the new flexbox tests,
>>> we're writing them against WebKit's code and they're changing as we go. In
>>> addition, all the properties are webkit prefixed. We're not writing test
>>> suites in isolation. We're writing them as we implement the feature or fix
>>> the appropriate bugs.
>>>
>>
>> Right, but then you'll need to unprefix those tests and reformat them in
>> order to upstream the tests anyway.
>>
>
> Yes, but the point is that it doesn't work to write the tests in the W3C
> repo first because what goes in the W3C repo doesn't work for us until we
> unprefix our implementation.
>

Yeah, so those tests should be kept separate from W3C test suites.

- Ryosuke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20111104/a2d9d5a8/attachment.html>