[webkit-dev] LayoutTest results choose from which folder?

[Dan Bernstein]

On Aug 23, 2009, at 11:28 AM, Dirk Pranke wrote:

>> Hi Chris,
>> In layout test results, we make the latest Mac OS X version the  
>> rule, and
>> earlier versions the exception. Tiger will look for results in mac- 
>> tiger
>> first, then in mac-leopard, then in mac-snowleopard, then in mac,  
>> then
>> finally in cross-platform results. Leopard will begin the search in
>> mac-leopard, continue to mac-snowleopard, then mac, the cross- 
>> platform.
>> As you can see, there are no expected results in mac-snowleopard  
>> (other than
>> the ones you just added), because it’s the latest Mac OS X version.  
>> We will
>> only start putting expected results in mac-snowleopard when the  
>> “latest”
>> version (for which we put results in mac) will be something  
>> different.
>> You should put the expected results for Snow Leopard in platform/ 
>> mac (or, if
>> they are cross-platform, alongside the test), and you should put  
>> the results
>> for Leopard and earlier in platform/mac-leopard.
>> —Dan
>
> Does this imply that if you've moved results from 'platform/mac' to
> 'platform/mac-leopard' when you switched from 10.5 to 10.6? (Since,
> presumable, some results that were in platform/mac were actually
> specific to 10.5?)

Yes, when the expected results of an existing test change under a new  
version of Mac OS X, legacy expected results are moved from platform/ 
mac to platform/mac-<last version with legacy behavior> and current  
expected results are put in platform/mac. <http://trac.webkit.org/changeset/47052 
 > is an example.

> I would've expected a different model, where if the output differed by
> version, then you had results in 'mac-leopard' and 'mac-snowleopard',
> and the presence in 'mac' meant it should be the same across both
> (this was the approach I was planning to use for Chromium, which is
> just starting to run into this problem with XP/Vista/Win 7).

Having platform/mac represent “latest and future versions of Mac OS X”  
means that when the next version appears, there is no need to do  
anything with tests whose results in the new version are the same as  
in the previous version. This model is analogous to how version- 
checking macros are used in WebKit code, by making exceptions for the  
past, not for the present: there is no BUILDING_ON_<latest version>  
macro.

> I can see
> some advantages to your approach, but it seems more confusing in the
> long run.

I hope my explanation clears up the confusion.

—Dan