[webkit-dev] Merging Skipped and test_expectations.txt formats WAS: Simplifying syntax in test_expectations.txt (bug 86691)

[Dirk Pranke]

On Thu, May 17, 2012 at 11:03 PM, Ojan Vafai <ojan at chromium.org> wrote:
> On Thu, May 17, 2012 at 10:37 PM, Maciej Stachowiak <mjs at apple.com> wrote:
>>
>>
>> On May 17, 2012, at 7:27 PM, Ojan Vafai <ojan at chromium.org> wrote:
>>
>> On Thu, May 17, 2012 at 4:29 PM, Maciej Stachowiak <mjs at apple.com> wrote:
>>>
>>>
>>> On May 17, 2012, at 1:42 PM, Ojan Vafai <ojan at chromium.org> wrote:
>>>
>>> On Thu, May 17, 2012 at 1:37 PM, Peter Kasting <pkasting at chromium.org>
>>> wrote:
>>>>
>>>> On Thu, May 17, 2012 at 1:34 PM, Ojan Vafai <ojan at chromium.org> wrote:
>>>>>
>>>>> 2. Make outcomes optional. If they are left out, then the test is
>>>>> skipped (unless the test is marked SLOW, in which case it's expected to
>>>>> pass). There is no SKIP modifier.
>>>>
>>>>
>>>> I don't think we should do this.  It seems very subtle.  I'd rather be
>>>> explicit.
>>>>
>>>> I'm OK with the rest of your numbered proposals.
>>>
>>>
>>> I disagree, but I'm fine with punting this to the list of controversial
>>> changes that we should discuss separately. FWIW, my main motivation here is
>>> that it allows us to unify the Skipped file format with the
>>> test_expectations.txt format. But again, we can discuss that separately.
>>>
>>>
>>> Adding SKIP (or whatever) to every line of skipped files is not a big
>>> hurdle, I think we could live with that is a transitions tep. I think the
>>> bigger hurdle is supporting chaining across multiple directories.
>>
>>
>> That's great. I don't think anyone is opposed to adding chaining and I
>> think that's on Dirk short-list of todos.
>>
>> The only potentially tricky thing here is figuring out what the platform
>> modifiers mean for non-Chromium ports, e.g. I imagine Qt will want similar
>> modifiers to Chromium (mac, linux, win, debug, release, etc). But I think
>> the difficulty here is more in getting the python code right than agreeing
>> on what the correct behavior is.
>>
>>
>> I think it would be good if platform modifiers in the expectations file
>> matched the platform names we use under the platform/ directory, either
>> literally or as a suffix. So for example "mac" in the chromium expectations
>> file could mean chromium-mac, in the qt expectations file it could mean
>> qt-mac, in the mac expectations file it should not be used, but snowleopard
>> would mean mac-snowleopard.
>
>
> That seems like a good way to organize it to me. Dirk, that make sense to
> you?
>

If I understand this properly, I think this is fine.

-- Dirk