[webkit-dev] Making changes to our copy of the W3C CSS1 test suite

[Darin Adler]

On Jun 2, 2010, at 11:29 AM, Eric Seidel wrote:

> What's the process for making changes to those files?

Here’s how I think we should do it:

    1) There’s one directory with a pristine copy of the W3C test suite, with no WebKit changes.
    2) If there are some tests that need to be fixed, fixed copies of those individual tests would go into another directory.
    3) The broken tests can be run as-is, and we can land expected results to reflect what the broken tests do.

Perhaps I would think differently about some aspect of this if we had introduced the “test expectations” concept for platforms other than Chromium.

> (Reminds me that we should move our "do not edit" layout tests to some special place.)

As far as reorganizing LayoutTests is concerned, I think we definitely need to do it.

The name of the directory itself is wrong, since they are not really tests of “layout” any more.

There should be some set of tests that are faster to run that omits the slower thorough tests. This was the original goal of “fast” but we have put tests outside “fast” more or less at random. Why are “editing” tests outside “fast”? Just the whim of the person who added them. Same comment on directories like “accessibility”.

The test suites that are imported from elsewhere should go into directories with clear names such as:

    ImportedTestSuites/W3C-CSS1

Rather than names like:

    css1

There are other categories of tests with special requirements. The ones in “http” and the ones in “websocket”, so those should go into clearly-named directories too.

But I am loathe to reorganize the tests when new-run-webkit-tests is still coming soon. I’d love to only have one test script to modify if we change things about the test organization that require changing the script.

    -- Darin