[webkit-dev] importing test suites (was: CSS 2.1 Test Suite)

[Ryosuke Niwa]

On Wed, Apr 11, 2012 at 3:52 PM, Jacob Goldstein <jacobg at adobe.com> wrote:

> +1 on not introducing new pixel tests and allowing someone other than the
> test author to create the -expected file.
>
> We may also be able to streamline some of this process by implementing
> some helper scripts.  Ultimately, someone will still have to review new
> files manually, but scripts should be able to speed up the process.
>

-1 on that. As I said on other threads about this topic, determining
whether a reference file adequately detect all bugs a test is intended to
test is hard, and losing the test coverage at the cost of lowering
maintenance cost is not necessary a good thing.

Also, adding a reference file would mean that either we're adding -ref.html
/ -noref.html files or modifying reftest.list. If doing the former, then we
can't use this approach in any directory where we use reftest.list at the
moment because we explicitly prohibit mixing naming convention and
reftest.list.

Modifying reftest.list is essentially modifying the test suite, and it
seems like there is a consensus that we don't want to do it.

- Ryosuke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120411/3f0a4d3a/attachment.html>