[webkit-dev] Importing W3C tests to webkit

[Maciej Stachowiak]

On May 23, 2012, at 3:13 PM, Dirk Pranke <dpranke at chromium.org> wrote:

> On Wed, May 23, 2012 at 2:30 PM, Maciej Stachowiak <mjs at apple.com> wrote:
>> 
>> Are you concerned just about the actual pixel results or also about keeping render tree dumps up to date?
> 
> Both are more maintenance than a text-only test. In my experience,
> maintaining pixel tests is more expensive, but I also don't have any
> experience maintaining a non-pixel-test-running port.

There are occasional changes that require rebaselining a very large number of render tree dumps. In my experience, this happens much less often than the frequency at which Chromium ports update pixel results, by several orders of magnitude.

> 
>> We can address the pixel result issue by introducing a new test that dumps its render tree but does not do pixel testing.
> 
> Yes, it might make sense to do this (although it's not obvious to me
> how we would do this without modifying the test sources). Maybe we
> would need to maintain a separate manifest or some other list
> somewhere to indicate which dirs or tests should include pixel
> results.

The quick and dirty way would be to add a SkipPixel directive (or whatever) to TestExpectations.

Cheers,
Maciej