[webkit-dev] Does NRWT let you indicate that a test should fail with a particular failure diff?
rniwa at webkit.org
Tue Jul 5 16:02:19 PDT 2011
On Tue, Jul 5, 2011 at 3:46 PM, Ojan Vafai <ojan at chromium.org> wrote:
> So, you could currently add a line like:
> BUGWK12345 : fast/canvas/canvastest.html = PASS
> We could simplify the syntax somewhat to not require the "= PASS" at the
How about renaming PASS to MATCH? MATCH will tell us that the actual result
matches the expected results but doesn't give us a false impression that the
test is "passing" with a correct output.
We could also change the bug format to be actual links instead (e.g.
> webkit.org/b/12345 and crbug.com/12345).
> webkit.org/b/12345 : fast/canvas/canvastest.html
> You can also list multiple bug per line:
> webkit.org/b/12345 webkit.org/b/33333 : fast/canvas/canvasttest.html
> That seem OK?
That seems like a huge improvement over the current BUGWK / BUGCR format.
Slightly OT: I'm increasingly convinced that we should try to find a scheme
> that is something in between the Chromium and the non-Chromium ports
> approaches and standardize on a simplified test_expectations format. That's
> certainly a separate discussion, but agreeing on this would be a step in
> that direction.
Yes! I'd love to have that discussion on a separate thread. The difference
between Chromium and non-Chromium ports is a huge pain when rebaselining
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the webkit-dev