[webkit-dev] CSS 2.1 Test Suite

[Ryosuke Niwa]

As we have previous discussed following
https://lists.webkit.org/pipermail/webkit-dev/2012-March/019782.html, it's
hard to judge whether a given reference result is enough to cover
everything the test intends to test. e.g. you can have a bug such that both
the test and the reference file ends up having the same rendering result.

- Ryosuke

On Tue, Apr 10, 2012 at 2:26 PM, Robert Hogan <lists at roberthogan.net> wrote:

> Hi there,
>
> We currently add tests from the CSS 2.1 suite as we fix them. They get
> added
> to the css2.1/20110323 folder in LayoutTests. Most of them don't have
> native reference tests yet in the suite so we (mostly I) have been adding
> homebrew reference results to the folder to avoid generating pixel results
> on all platforms. (see http://webkitmemes.tumblr.com/post/20788159625 !)
>
> These reference-results are easily removed once superseded but it might be
> cleaner just to move them, and the associated css tests, to a folder of
> their own in fast/css. That will allow css2.1/20110323 to be a clean import
> that the 8500 or so passing tests can be added to in 20 or 30 batches.[1]
> It will also make NRWT's reftests harness work with the suite.
>
> Does anyone object to that approach? The only thing going against it seems
> to be the principle that imported tests should be stored separately and
> together but this is more a case of using them to fix bugs and prevent
> future regressions while allowing a clean import of the CSS 2.1 test suite
> to take place in parallel.
>
> The problem this does not solve is how we avoid creating pixel results for
> tests that already pass but which do not have reftests of their own. Again
> I would be in favour of putting these in fast/css and keeping them there
> until reftests are added to the suite. This would allow us to prevent them
> regressing and come up with a reftest for them that can be submitted to the
> CSS test suite guys.
>
> The end result would be that we only directly import to the css2.1 folder
> those tests in the CSS test suite that have ref tests native to the suite.
>
> Thanks,
> Robert
>
>
> [1] I keep a local and relatively up to date copy of the passing and
> failing tests in separate folders in my checkout.  Yes I know I should
> create bugs for them and get started on landing the passes.
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120410/b65e21d6/attachment.html>