[webkit-dev] summarizing the current proposals for changing the test_expectations syntax (and semantics)

[Dirk Pranke]

Per Maciej's request, I will attempt to summarize both the decisions
reached in the prior mega-thread, and the discussion that has so far
occurred subsequently occurred on webkit.org/b/86749. This is largely
requoting one of Ojan's earlier messages:

Changes mostly agreed to:

* WONTFIX will imply SKIP

* we will now indicate the bug associated with an expectation by using
an abbreviated URL, e.g. webkit.org/b/12345. You will still be able to
associate a line with a user instead, but we will shift to
bug(ircnick).

* SKIP, SLOW, TIMEOUT, and WONTFIX will move to the right of the test
name, leaving just platform/configuration modifiers on the left

So, examples:

webkit.org/b/12345 LION DEBUG : foo/bar.html = TEXT
bug(ojan) : foo/baz.html = SLOW CRASH

Changes still being discussed:

* We will keep some sort of mechanism for delimiting the different
parts of the line, but we're open to changing what the delimiters are.
Current leading favorites are using "\" in between the platform
modifiers, the test, and the expected outcome, and using "[" and "]"
to bracket the text, e.g.:

webkit.org/b/12345 LION DEBUG \ foo.bar.html \ TEXT
bug(ojan) \ foo.baz.html \ SLOW CRASH

and:

webkit.org/b/12345 LION DEBUG [ foo.bar.html ] TEXT
bug(ojan) [ foo.baz.html ] SLOW CRASH

I have officially waffled in favor of either of these, but currently,
I'm not sure I prefer either of the two to the existing syntax. I
would, however, strongly prefer not to use "," as a delimiter, because
I don't want to make the syntax looks like a CSV file, and I think
it's probably not a great idea to use any character that might
actually occur in a test name.

* Should we change the case of the keywords to either all lower case,
or mixed case?

I am open to all lower case, since we still have delimiters. I don't
think mixed case will improve readability over either all lower or all
upper.

* Should we change the outcome modifiers to something else and/or
change their semantics?

That's being discussed in a separate thread. I have also filed
webkit.org/b/86892 to track both of these issues.

* Should we just get rid of "SLOW" ?

I have filed webkit.org/b/86889 for this one.

-- Dirk