[webkit-dev] TestExpectations syntax changes, last call (for a while, at least) ...

[Ryosuke Niwa]

On Wed, Jun 13, 2012 at 4:32 PM, Dirk Pranke <dpranke at chromium.org> wrote:

> On Wed, Jun 13, 2012 at 4:26 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:
> > On Wed, Jun 13, 2012 at 4:12 PM, Benjamin Poulain <benjamin at webkit.org>
> > wrote:
> >> On Wed, Jun 13, 2012 at 3:53 PM, Dirk Pranke <dpranke at chromium.org>
> wrote:
> >> > webkit.org/12345 WIN MAC DEBUG \
> >> > animations/stop-animation-on-suspend.html \ CRASH TEXT PASS
> >>
> >> My bikeshedding:
> >> -In my opinion, the backslash is not any better than the old
> >> delimiter. It looks like we are escaping something.
> >>
> >> I liked better the proposal to have platform and results grouped
> >> instead of the delimiter.
> >> e.g: webkit.org/12345 (WIN MAC DEBUG)
> >> animations/stop-animation-on-suspend.html (CRASH TEXT PASS)
> >>
> >> -I prefer lowercase keywords, possibly with the first character
> >> uppercase (Crash, Text, Pass, Fail, Win, Mac, Debug). Do we use all
> >> uppercase keywords somewhere else in the project?
> >
> > So something like
> > webkit.org/12345 [Win Mac
> Debug] animations/stop-animation-on-suspend.html
> > [Crash Text Pass]
> > ? I like that!
>
> This seems quite readable to me as well. As mentioned above, I'd be
> inclined to make the absence of [] on the right hand side mean "SKIP".
>

That sounds very reasonable.

In fact, I'd argue that it should be the only way to skip tests. It makes
no sense to have any other test expectation when a test is skipped because
the types of failures of skipped tests (when actually ran) will very likely
change over time, and there is no way to track them.

- Ryosuke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120613/b685ef75/attachment.html>