[webkit-dev] Does NRWT let you indicate that a test should fail with a particular failure diff?

[Ryosuke Niwa]

On Tue, Jul 5, 2011 at 5:40 PM, Dirk Pranke <dpranke at chromium.org> wrote:
>
> I keep hearing that the syntax is "excessively complicated". It's a
> pretty simple syntax, but even you think that it is complicated, but
> in what way is it excessively so, given that we actively use all of
> the features it supports?
>

I find having to type : and = and the ordering of tokens extremely annoying.
 Given no token can be repeated, why can't we just have a set of
space-separated tokens?

e.g.
BUGCR88230 VISTA : fast/dom/dom-parse-serialize-display.html = PASS TIMEOUT
can just be:
BUGCR88230 VISTA fast/dom/dom-parse-serialize-display.html PASS TIMEOUT
or any of the following (not exhaustive):
BUGCR88230 VISTA PASS TIMEOUT fast/dom/dom-parse-serialize-display.html
BUGCR88230 PASS TIMEOUT VISTA fast/dom/dom-parse-serialize-display.html
PASS TIMEOUT BUGCR88230 VISTA fast/dom/dom-parse-serialize-display.html
PASS TIMEOUT VISTA BUGCR88230 fast/dom/dom-parse-serialize-display.html

On Tue, Jul 5, 2011 at 6:00 PM, Ojan Vafai <ojan at chromium.org> wrote:
>
> I think people are more confused with the various failure types. If we move
> to a model where we check in failing expectations and just list them with
> the bug number, I would be fine with going back to a world without
> fine-grained failure types (i.e. we'd just have PASS, FAIL, TIMEOUT and
> CRASH).
>
> People are also confused by dealing with the multitude of different
> platforms, but I don't have concrete suggestions on how to improve that
> situation.
>
> Finally, people are confused by how SLOW works. I'd much rather we just
> increase the default timeout and give a shorter timeout for tests that are
> listed as having TIMEOUT expectations. That maintains the benefits to bot
> cycle time without needing to manually maintain a list of slow tests.
>

+1 for getting rid of SLOW, IMAGE, TEXT, and IMAGE+TEXT.

- Ryosuke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20110705/03894e76/attachment.html>