[webkit-dev] the new TestExpectations syntax is landing soon

[Osztrogonac Csaba]

Unfortunately r129047 broke the results.html, see https://bugs.webkit.org/show_bug.cgi?id=96845#c9 for details.

Dirk Pranke írta:
> These changes are now starting to land ...
> 
> as of r129047, TEXT, IMAGE+TEXT, and AUDIO are no longer legal
> keywords in the TestExpectations syntax ... you should use "FAIL"
> instead.
> 
> I will be landing the support for the new syntax as quickly as I can
> to minimize the transition period. Apologies for the inconvenience.
> 
> -- Dirk
> 
> On Wed, Sep 12, 2012 at 4:29 PM, Dirk Pranke <dpranke at chromium.org> wrote:
>> Hi all,
>>
>> The new format of the much-debated TestExpectations syntax will be
>> landing soon (hopefully in the next couple days).
>>
>> For those of who have forgotten / repressed the earlier debates, the
>> new syntax looks something like:
>>
>> webkit.org/b/12345 [ Mac Vista] fast/html/keygen.html [ ImageOnlyFailure ]
>>
>> Andis documented in full at
>> https://trac.webkit.org/wiki/TestExpectations#NewSyntaxNotquiteyetlanded
>> .
>>
>> ( The "[" and "]" characters are delimiters, not EBNF "optional"
>> markers, although those sections are in fact optional :) ).
>>
>> Note that the new syntax means that Skipped files are a syntactic
>> subset of TestExpectations files, and I plan to convert all of the
>> Skipped files to TestExpectations files via copy and paste shortly
>> after the new syntax is landed, and then drop support for Skipped
>> files (I will update ORWT to use the new files and treat any entry as
>> a Skip).
>>
>> The plan for landing these changes is:
>>
>> 1) Add support for parsing the new lines and converting them back into
>> the old format (internally) so that both syntaxes are supported
>> 2) Convert all the existing files over
>> 3) Make sure things aren't broken :)
>> 4) Drop support for the old syntax
>>
>> I plan for this to all happen quickly, in less than a day. This means
>> that if you have patches posted that modify those files they may
>> become stale and need to be updated.
>>
>> Changes from the old syntax:
>>
>> 1. We use URLs (a specific whitelisted set; let me know if you want to
>> add to it) instead of BUGWK12345 etc.
>> 2. We use "bug(dpranke) instead of BUGDPRANKE
>> 3. We use CamelCase instead of SHOUTING
>> 4. We use Failure to represent what used to be TEXT, IMAGE+TEXT, and
>> AUDIO - these failures will be indistinguishable in the new world,
>> meaning that you can't distinguish between "text only" and "both image
>> and text". Since only Chromium runs pixel tests by default, this
>> shouldn't be a big deal.
>> 5. We use ImageOnlyFailure to represent what used to be IMAGE
>> 6. We use "[" and "]" for delimiters instead of ":" and "="
>> 7. We use "#" instead of "//" as a comment
>> 7. WontFix will now imply Skip, i.e., tests marked WontFix will
>> automatically be Skipped
>> 8. WontFix and Skip will not require (or even allow) any other
>> expectations, i.e., you can't say "[ WontFix Crash ]". If you want to
>> indicate that the test will crash if you actually run it, use a
>> comment.
>> 9. WontFix, Skip, Slow, and Rebaseline all move from the left hand
>> side to the right. The only keywords on the left restrict which
>> configurations the lines apply to.
>>
>> I will send out follow-up emails as this stuff lands. Please let me
>> know if you have any questions. Thanks!
>>
>> -- Dirk
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo/webkit-dev