[webkit-dev] Should we close the tree? (was: Re: the new TestExpectations syntax is landing soon)

[Alexey Proskuryakov]

Now tracked as <https://bugs.webkit.org/show_bug.cgi?id=97182>.

I think that we should close the tree if resolving this takes any significant time. Not being able to see how exactly tests are failing on other platforms is unacceptable.

- WBR, Alexey Proskuryakov


20.09.2012, в 3:54, Osztrogonac Csaba <oszi at inf.u-szeged.hu> написал(а):

> Unfortunately r129047 broke the results.html, see https://bugs.webkit.org/show_bug.cgi?id=96845#c9 for details.
> 
> Dirk Pranke írta:
>> These changes are now starting to land ...
>> as of r129047, TEXT, IMAGE+TEXT, and AUDIO are no longer legal
>> keywords in the TestExpectations syntax ... you should use "FAIL"
>> instead.
>> I will be landing the support for the new syntax as quickly as I can
>> to minimize the transition period. Apologies for the inconvenience.
>> -- Dirk
>> On Wed, Sep 12, 2012 at 4:29 PM, Dirk Pranke <dpranke at chromium.org> wrote:
>>> Hi all,
>>> 
>>> The new format of the much-debated TestExpectations syntax will be
>>> landing soon (hopefully in the next couple days).
>>> 
>>> For those of who have forgotten / repressed the earlier debates, the
>>> new syntax looks something like:
>>> 
>>> webkit.org/b/12345 [ Mac Vista] fast/html/keygen.html [ ImageOnlyFailure ]
>>> 
>>> Andis documented in full at
>>> https://trac.webkit.org/wiki/TestExpectations#NewSyntaxNotquiteyetlanded
>>> .
>>> 
>>> ( The "[" and "]" characters are delimiters, not EBNF "optional"
>>> markers, although those sections are in fact optional :) ).
>>> 
>>> Note that the new syntax means that Skipped files are a syntactic
>>> subset of TestExpectations files, and I plan to convert all of the
>>> Skipped files to TestExpectations files via copy and paste shortly
>>> after the new syntax is landed, and then drop support for Skipped
>>> files (I will update ORWT to use the new files and treat any entry as
>>> a Skip).
>>> 
>>> The plan for landing these changes is:
>>> 
>>> 1) Add support for parsing the new lines and converting them back into
>>> the old format (internally) so that both syntaxes are supported
>>> 2) Convert all the existing files over
>>> 3) Make sure things aren't broken :)
>>> 4) Drop support for the old syntax
>>> 
>>> I plan for this to all happen quickly, in less than a day. This means
>>> that if you have patches posted that modify those files they may
>>> become stale and need to be updated.
>>> 
>>> Changes from the old syntax:
>>> 
>>> 1. We use URLs (a specific whitelisted set; let me know if you want to
>>> add to it) instead of BUGWK12345 etc.
>>> 2. We use "bug(dpranke) instead of BUGDPRANKE
>>> 3. We use CamelCase instead of SHOUTING
>>> 4. We use Failure to represent what used to be TEXT, IMAGE+TEXT, and
>>> AUDIO - these failures will be indistinguishable in the new world,
>>> meaning that you can't distinguish between "text only" and "both image
>>> and text". Since only Chromium runs pixel tests by default, this
>>> shouldn't be a big deal.
>>> 5. We use ImageOnlyFailure to represent what used to be IMAGE
>>> 6. We use "[" and "]" for delimiters instead of ":" and "="
>>> 7. We use "#" instead of "//" as a comment
>>> 7. WontFix will now imply Skip, i.e., tests marked WontFix will
>>> automatically be Skipped
>>> 8. WontFix and Skip will not require (or even allow) any other
>>> expectations, i.e., you can't say "[ WontFix Crash ]". If you want to
>>> indicate that the test will crash if you actually run it, use a
>>> comment.
>>> 9. WontFix, Skip, Slow, and Rebaseline all move from the left hand
>>> side to the right. The only keywords on the left restrict which
>>> configurations the lines apply to.
>>> 
>>> I will send out follow-up emails as this stuff lands. Please let me
>>> know if you have any questions. Thanks!
>>> 
>>> -- Dirk
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo/webkit-dev
> 
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo/webkit-dev




- WBR, Alexey Proskuryakov