[webkit-dev] Supporting w3c ref tests and changing our convention

[Tor Arne Vestbø]

On 07.11.11 15:44, Alan Stearns wrote:
> What if we defer some of the W3C metadata work until tests were actually
> submitted to the W3C?
>
> 1. Tests we pull from W3C can run from manifests, since they are provided.
>
> 2. Tests we develop ourselves just use a naming convention (refs are named
> *-ref.html, and there's one ref per test even if that's duplicative)
>
> 3. When we choose to share a set of tests with the W3C, we do the extra work
> of adding metadata to the tests and possibly refactoring to reduce the
> number of -ref files. Once the W3C approves the tests we pull their copies
> and delete ours.

I think this is the "best of both worlds" approach, in that it's easy to 
import W3C tests (manifest is already there), easy to add new tests in 
WebKit (no need to regenerate the manifest when landing, or on every 
build), and gives a clear way of what needs to be done when upstreaming 
tests to W3C.

The link-approach seems the least ideal, as it puts more strain on each 
port to fix their DRT, versus adding features to the shared test 
scripts, and going manifest-only adds more process for adding new tests 
-- a process that I think we can defer and batch up to when we want to 
upstream tests.

Tor Arne