[webkit-dev] Does NRWT let you indicate that a test should fail with a particular failure diff?

[Dirk Pranke]

On Tue, Jul 5, 2011 at 6:12 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:
> On Tue, Jul 5, 2011 at 5:40 PM, Dirk Pranke <dpranke at chromium.org> wrote:
>>
>> I keep hearing that the syntax is "excessively complicated". It's a
>> pretty simple syntax, but even you think that it is complicated, but
>> in what way is it excessively so, given that we actively use all of
>> the features it supports?
>
> I find having to type : and = and the ordering of tokens extremely annoying.
>  Given no token can be repeated, why can't we just have a set of
> space-separated tokens?
> e.g.
> BUGCR88230 VISTA : fast/dom/dom-parse-serialize-display.html = PASS TIMEOUT
> can just be:
> BUGCR88230 VISTA fast/dom/dom-parse-serialize-display.html PASS TIMEOUT
> or any of the following (not exhaustive):
> BUGCR88230 VISTA PASS TIMEOUT fast/dom/dom-parse-serialize-display.html
> BUGCR88230 PASS TIMEOUT VISTA fast/dom/dom-parse-serialize-display.html
> PASS TIMEOUT BUGCR88230 VISTA fast/dom/dom-parse-serialize-display.html
> PASS TIMEOUT VISTA BUGCR88230 fast/dom/dom-parse-serialize-display.html

Thanks for the (specific!) feedback :)

I personally find your examples to be much harder to parse visually.
Partially the advantage to putting the test in the middle of the line
is that it makes it easy to separate the "stuff on the left" from the
"expectations" on the right that actually describe what the result of
the test is supposed to be.

I grant that it can be unclear which side of the test certain
expectations show up on (e.g., SLOW, SKIP). I personally also think
that it's annoying to have to specify expectations if you are also
skipping the test. One could move the "SKIP" to the right hand side,
but it seems a bit weird to call "SKIP" an expected result, since the
test isn't actually running.

-- Dirk