[webkit-dev] LayoutTests results fallback graph

[Mark Rowe]

On 2011-07-10, at 10:52, Adam Barth wrote:

> Hi webkit-dev,
> 
> In trying to understand how our LayoutTest results system works, I've
> created a digram of the fallback graph among the various
> platform-specific directories:
> 
> https://docs.google.com/drawings/d/1z65SKkWrD4Slm6jobIphHwwRADyUtjOAxwGBVKBY8Kc/edit?hl=en_US
> 
> Unfortunately, the fallback graph is not a tree, as one might imagine
> initially.  I'd like to propose two small changes, which will
> hopefully make the system more sensible globally.  I'm happy to do all
> the work required to make these changes:
> 
> 1) The "win" port should fall back either to "all" (the platform
> independent results) or to "mac," but not to "mac-snowleopard", as it
> does currently.  (I slightly prefer "all", but "mac" would also be
> fine with me.)

I'd argue that falling back to "mac" doesn't make any sense.  The regression tests are run against a WebKit using the latest shipping Safari's version of the underlying dependencies.  That almost always corresponds to the latest shipping Mac OS X release's components, and not to the components from future versions of Mac OS X.

I also think that falling back to "all" would only be advisable if we were to also switch Windows DRT away from the legacy Mac-style form controls and rebaseline all of the Windows test results. We've shied away from this in the past because it would result in a big increase in the number of Windows-specific results.

- Mark