[webkit-dev] LayoutTests results fallback graph

[Adam Barth]

On Sun, Jul 10, 2011 at 12:11 PM, Adam Barth <abarth at webkit.org> wrote:
> On Sun, Jul 10, 2011 at 12:01 PM, James Robinson <jamesr at google.com> wrote:
>> On Jul 10, 2011 10:53 AM, "Adam Barth" <abarth at webkit.org> wrote:
>>> Hi webkit-dev,
>>>
>>> In trying to understand how our LayoutTest results system works, I've
>>> created a digram of the fallback graph among the various
>>> platform-specific directories:
>>> https://docs.google.com/drawings/d/1z65SKkWrD4Slm6jobIphHwwRADyUtjOAxwGBVKBY8Kc/edit?hl=en_US
>>>
>>> Unfortunately, the fallback graph is not a tree, as one might imagine
>>> initially.  I'd like to propose two small changes, which will
>>> hopefully make the system more sensible globally.  I'm happy to do all
>>> the work required to make these changes:
>>>
>>> 1) The "win" port should fall back either to "all" (the platform
>>> independent results) or to "mac," but not to "mac-snowleopard", as it
>>> does currently.  (I slightly prefer "all", but "mac" would also be
>>> fine with me.)
>>>
>>> 2) The "chromium" port should fall back directly to "all" rather than
>>> taking a detour through various Apple-maintained ports, as it does
>>> currently.
>>>
>>> These changes have the following virtues:
>>>
>>> A) The resulting fallback graph will be a tree, making the fallback
>>> graph easier to understand for both humans and automated tools.
>>> B) The Chromium port will behave more like the other ports (e.g., GTK
>>> and Qt), rather than being a parasite on Apple-maintained ports.
>>>
>>> These changes might increase the number of image baselines we store in
>>> the tree for "chromium-mac"-derived ports (because there will be fewer
>>> redundant fallback paths), but I expect that cost to be relatively
>>> small because essentially every port has different image baselines
>>> anyway
>>
>> Could you measure this? I suspect that not falling back on the mac pixel
>> results will mean checking in a few thousand more pngs, but that's just a
>> guess.
>
> That seems possible:
>
> abarth at quadzen:~/git/webkit/LayoutTests/platform/chromium-mac$ find .
> -name "*.png" | wc -l
>     900
> abarth at quadzen:~/git/webkit/LayoutTests/platform/mac$ find . -name
> "*.png" | wc -l
>    6688
> abarth at quadzen:~/git/webkit/LayoutTests/platform/chromium-win$ find .
> -name "*.png" | wc -l
>    5988
> abarth at quadzen:~/git/webkit/LayoutTests/platform/chromium-linux$ find
> . -name "*.png" | wc -l
>    5731
>
> However, I would expect those savings to disappear once we finish
> moving chromium-mac to Skia.

More numbers for scale:

abarth at quadzen:~/git/webkit/LayoutTests/platform/qt$ find . -name
"*.png" | wc -l
    3852
abarth at quadzen:~/git/webkit/LayoutTests/platform$ find chromium* -name
"*.png" | wc -l
   14653
abarth at quadzen:~/git/webkit/LayoutTests/platform$ find . -name "*.png" | wc -l
   35350

Adam