[webkit-dev] LayoutTests results fallback graph

[Adam Barth]

On Sun, Jul 10, 2011 at 12:38 PM, Mark Rowe <mrowe at apple.com> wrote:
> On 2011-07-10, at 10:52, Adam Barth wrote:
>> Hi webkit-dev,
>>
>> In trying to understand how our LayoutTest results system works, I've
>> created a digram of the fallback graph among the various
>> platform-specific directories:
>>
>> https://docs.google.com/drawings/d/1z65SKkWrD4Slm6jobIphHwwRADyUtjOAxwGBVKBY8Kc/edit?hl=en_US
>>
>> Unfortunately, the fallback graph is not a tree, as one might imagine
>> initially.  I'd like to propose two small changes, which will
>> hopefully make the system more sensible globally.  I'm happy to do all
>> the work required to make these changes:
>>
>> 1) The "win" port should fall back either to "all" (the platform
>> independent results) or to "mac," but not to "mac-snowleopard", as it
>> does currently.  (I slightly prefer "all", but "mac" would also be
>> fine with me.)
>
> I'd argue that falling back to "mac" doesn't make any sense.  The regression tests are run against a WebKit using the latest shipping Safari's version of the underlying dependencies.  That almost always corresponds to the latest shipping Mac OS X release's components, and not to the components from future versions of Mac OS X.

There appears to be almost zero practical different between "win"
falling back to "mac" versus falling back to "mac-snowleopard":

abarth at quadzen:~/git/webkit/LayoutTests/platform$ find mac -name "*.png" | wc -l
    6688
abarth at quadzen:~/git/webkit/LayoutTests/platform$ find
mac-snowleopard/ -name "*.png" | wc -l
       3

> I also think that falling back to "all" would only be advisable if we were to also switch Windows DRT away from the legacy Mac-style form controls and rebaseline all of the Windows test results. We've shied away from this in the past because it would result in a big increase in the number of Windows-specific results.

I suspect there's actually an invisible "apple" port (which is a peer
to "chromium", "gtk", and "qt") that should hold those common
baselines.

Adam