[webkit-dev] TestExpectations syntax changes, last call (for a while, at least) ...

[Dirk Pranke]

Hi all,

Because I have infinite patience for bikeshedding, I thought I would
send out Yet Another note on the proposed changes to the expectation
syntax.

Based on the last thread, I'm planning to change ORWT so that it will
recognize the syntax in the TestExpectations files and treat any
non-PASS entry as Skipped, so we will be able to switch over to
TestExpectations files wholesale, so you might want to care about this
...

I would also like to batch all of the changes up at once and then
defer any changes for a month or more, just so the syntax isn't
changing frequently and we have to retrain everyone. I'm of course
open to changing things whenever we need to :).

The current syntax is documented here:

http://trac.webkit.org/wiki/TestExpectations

In https://bugs.webkit.org/show_bug.cgi?id=86796 , we are proposing to
change it to something like:

webkit.org/12345 WIN MAC DEBUG \
animations/stop-animation-on-suspend.html \ CRASH TEXT PASS

* the left hand side of the test now just contains the bug link and
keywords that modify which configurations the line applies to.

* You may use bug(dpranke) instead of a URL on the left hand side of
the expression.

* The REBASELINE, SLOW, SKIP, WONTFIX keywords will move to the
right-hand side (note that REBASELINE doesn't ever actually get
checked in).

* we use "\" (backslash) as a delimiter instead of ":" and "="

* The FAIL keyword is being removed as we speak.

* We'll probably rename IMAGE+TEXT to IMAGE_AND_TEXT.

* WONTFIX will have the same effect as SKIP but has a different
meaning: we don't plan on passing the test at all; SKIP is intended to
work around temporary bugs or problems.

* As of now the keywords remain all UPPERCASE. I'm happy to change
them to all lowercase or mixed case if there's a consensus that such a
change is desired.

Speak now if you really don't like these proposals, or hold your peace
for at least a month ;)

-- Dirk