[webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

[Ryosuke Niwa]

Correction: the person from Opera who tried to add reference files
is Geoffrey Sneddon as Anne pointed out.

+Ms2ger, +ehsan from Mozilla, and +jgraham from Opera.

Also note that both Ehsan and James have pointed out that my concern that a
test and the corresponding ref will match for wrong reasons (e.g. the
entire page is blank) could be be mitigated by having good noref
(expected-mismatch in WebKit convention). James also noted that they
haven't had such problems in practice.

Given their experience and knowledge working with reftest, I change my
position and will support writing our own reference files as well. But I'm
still interested in moving forward with upstreaming our reference files to
the W3C repository and making clear distinction between ones we import and
ones we write ourselves.

- Ryosuke

On Thu, Apr 12, 2012 at 1:45 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:

> I've done more digging and talked with folks from Mozilla and Opera about
> this topic. Here's a summary.
>
> Daniel Glazman from Opera has tried adding reference files for W3C's:
> http://lists.w3.org/Archives/Public/public-css-testsuite/2010Sep/0030.html
>
> However, some people in the CSS working group opposed to make this change
> in last minute because it would have delayed REC, and other parties did not
> support the proposal enough to convince the WG as a whole to import the
> reference files.
> (See the member only mailing list around September 2010 if you have
> access).
>
> Ms2ger also has more reference files available at
> https://bitbucket.org/ms2ger/css-tests
>
> On the other hand, CSS3 test suite consists entirely of reference tests
> and testharness.js tests (js text based tests). Given that and TabAtkins
> and other folks at the CSS WG now prefer reference tests over pixel tests
> in addition to the fact CSS2.1 has already been REC'ed, it might be
> possible for us to convince the WG to convert the existing tests now.
>
> In my opinion, we should definitely try to make changes upstream because
> then virtually all browser vendors will benefit.
>
> - Ryosuke
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120412/7a9a5537/attachment.html>