[webkit-dev] CSS 2.1 Test Suite

[Robert Hogan]

Hi there,

We currently add tests from the CSS 2.1 suite as we fix them. They get added 
to the css2.1/20110323 folder in LayoutTests. Most of them don't have 
native reference tests yet in the suite so we (mostly I) have been adding 
homebrew reference results to the folder to avoid generating pixel results 
on all platforms. (see http://webkitmemes.tumblr.com/post/20788159625 !)

These reference-results are easily removed once superseded but it might be 
cleaner just to move them, and the associated css tests, to a folder of 
their own in fast/css. That will allow css2.1/20110323 to be a clean import 
that the 8500 or so passing tests can be added to in 20 or 30 batches.[1] 
It will also make NRWT's reftests harness work with the suite.

Does anyone object to that approach? The only thing going against it seems 
to be the principle that imported tests should be stored separately and 
together but this is more a case of using them to fix bugs and prevent 
future regressions while allowing a clean import of the CSS 2.1 test suite 
to take place in parallel.

The problem this does not solve is how we avoid creating pixel results for 
tests that already pass but which do not have reftests of their own. Again 
I would be in favour of putting these in fast/css and keeping them there 
until reftests are added to the suite. This would allow us to prevent them 
regressing and come up with a reftest for them that can be submitted to the 
CSS test suite guys.

The end result would be that we only directly import to the css2.1 folder 
those tests in the CSS test suite that have ref tests native to the suite.

Thanks,
Robert


[1] I keep a local and relatively up to date copy of the passing and 
failing tests in separate folders in my checkout.  Yes I know I should 
create bugs for them and get started on landing the passes.