[webkit-dev] Does NRWT let you indicate that a test should fail with a particular failure diff?

Ojan Vafai ojan at chromium.org
Tue Jul 5 17:16:13 PDT 2011

On Tue, Jul 5, 2011 at 4:51 PM, Dirk Pranke <dpranke at chromium.org> wrote:

> On Tue, Jul 5, 2011 at 1:53 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:
> > On Jul 5, 2011 1:26 PM, "Dirk Pranke" <dpranke at chromium.org> wrote:
> >> > However, we can do the same with the existing testing framework since
> we
> >> > can
> >> > associate a test with a bug by adding a line like this:
> >> > BUGWK????? my-test.html = PASS
> >>
> >> You lost me here ...
> >
> > What I meant is that we can use test_expectations.txt to annotate a test
> > with a bug number even if we overrode -expected.txt by adding that line.
> > This allows us to track of tests with bad expected results while also
> > allowing us to catch any changes in actual result.
> >
> Ah. That makes a lot of sense. Good idea.
> On Tue, Jul 5, 2011 at 3:46 PM, Ojan Vafai <ojan at chromium.org> wrote:
> > We could simplify the syntax somewhat to not require the "= PASS" at the
> > end. We could also change the bug format to be actual links instead (e.g.
> > webkit.org/b/12345 and crbug.com/12345).
> > webkit.org/b/12345 : fast/canvas/canvastest.html
> > You can also list multiple bug per line:
> > webkit.org/b/12345 webkit.org/b/33333 : fast/canvas/canvasttest.html
> > That seem OK?
> I'm not so much a fan of this change. It's more typing and I'm not
> sure if it makes anything much easier for the user (maybe viewing in a
> text editor that will automatically hyperlink the text, I guess).
> Note that the current syntax does support multiple bugs per line
> (although not the hyperlink syntax you propose); I'm sure you knew
> that, so perhaps you were just mentioning that for the others on the
> list.

One of the problems with the current format is that it's excessively
complicated. Avoiding magic syntax and just using URLs makes it so there is
less specialized syntax to make sense of. It also has the potential
advantages you mention (e.g. you could copy-paste the text into your
browser's address bar), but that's secondary.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20110705/69c5c156/attachment-0001.html>

More information about the webkit-dev mailing list