[webkit-dev] Simplifying syntax in test_expectations.txt (bug 86691)

[Ryosuke Niwa]

On Wed, May 16, 2012 at 10:39 PM, Dirk Pranke <dpranke at chromium.org> wrote:

> On Wed, May 16, 2012 at 10:30 PM, Maciej Stachowiak <mjs at apple.com> wrote:
> > On May 16, 2012, at 9:08 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:
> > There has been some complaints / discussions about how syntax in
> > test_expectations.txt is confusing (and I agree with you) on webkit-dev
> and
> > at contributors' meeting.
> >
> > So I have a patch on https://bugs.webkit.org/show_bug.cgi?id=86691 to
> > simplify syntax in test_expectation.txt as follows:
> >
> > Bug modififers are bug numebr themselves for WebKit and URLs for
> non-WebKit
> > bugs
> > e.g. 12345 and crbug.com/12345 instead of BUGWK12345 and BUGCR12345
> > respectively
> >
> > I think I might prefer webkit.org/b/12345 - it's a bit less concise but
> the
> > meaning is more contextually obvious and you can cut and paste it into a
> > browser address field to follow the link, rather than having to know what
> > site to enter it into.
> >
> > : and = delimiters are no longer needed
> > All modifiers and expectations show before test name
> > e.g. 12345 WIN MAC TEXT IMAGE test.html instead of BUGWK12345 WIN MAC :
> > test.html = TEXT IMAGE
> >
> > Would it be possible to also change the modifiers to lower case or mixed
> > case? All-caps is hard to type and uncomfortable to read. As for where
> the
> > modifiers go, was there a logical basis for the previous separation into
> two
> > groups? Would we be losing something by doing that?
> >
>
> There was a semi-logical basis, in that the stuff on the right of the
> test clearly specified the outcome of the test (PASS, IMAGE, TEXT,
> etc.). The stuff on the left was less well defined: there's the bug
> numbers, the platform/configuration info (MAC LINUX RELEASE, etc.),
> and some other stuff that there is less of a good place for (SLOW,
> SKIP, WONTFIX).
>

What I find most annoying with the current syntax is that modifiers and
expectations appear on the opposite ends of a test name. So if I'm copying
& pasting a test name to add a new entry to test_expectations.txt, then I
would have to type something on both ends instead of typing stuff in before
or after pasting the test name.

Taking Benjamin's idea to make the meaning of tokens explicit and Maciej's
(and my original) proposal to use webkit.org/b/12345, how about something
like
webkit.org/12356 [Win Mac] [Text] failing-test.html
<rniwa> [Linux] [Skip] temporarily-skipped-test.html
Here, platforms and expectations are wrapped around [] and non-URL comments
are wrapped in <>. (I think we should just use IRC nick).

- Ryosuke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120516/1f56c2c5/attachment.html>