[webkit-dev] generic test expectations?

[Glenn Adams]

On Tue, Nov 13, 2012 at 12:09 PM, Dirk Pranke <dpranke at chromium.org> wrote:

> We don't currently support port-specific reftests (or at least, not
> very well), and we certainly should be trying to minimize where they
> occur.


Hmm, I actually used port specific reftest expectation files in a recent
patch [1] (since rolled out), and it appeared to work (as a way to
effectively rebaseline those expectations). So something seems to be
working.

[1] http://trac.webkit.org/changeset/133529


> At any rate, we encourage people these days to check in expected
> failures rather than suppressing them using the TestExpectations
> files.


The problem is essentially a chicken and egg problem. I don't know what the
per-port failures will be ahead of time, but I do know the set of "correct"
expectations. Since I am (independently) unable to build/test all ports run
by build bots, I would like to commit the set of tests plus known good
expectations as a preliminary step (with a generic skip all tests for all
ports), and then subsequently commit the feature itself, and then
subsequently override the generic skip on a port specific basis,
effectively re-enabling the tests on a port by port basis as I refine the
feature patch (as needed) to handle port specific behavioral differences.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20121113/1c5479d7/attachment.html>