[webkit-dev] LayoutTests results fallback graph

[Eric Seidel]

I believe what Adam means by this (it wasn't immediately clear to me),
is that we have 1500 redundant result files with duplicate git hashes
to some other file.  This could be calculated by the
deduplicate_results.py script by removing any of the current fallback
logic and just look at raw duplicates.

Please correct me if I've misunderstood.

-eric

On Wed, Jul 13, 2011 at 2:06 PM, Adam Barth <abarth at webkit.org> wrote:
> On Wed, Jul 13, 2011 at 1:56 PM, Adam Barth <abarth at webkit.org> wrote:
>> On Wed, Jul 13, 2011 at 1:36 PM, Dirk Pranke <dpranke at chromium.org> wrote:
>>> On Wed, Jul 13, 2011 at 1:28 PM, Adam Barth <abarth at webkit.org> wrote:
>>>> On Wed, Jul 13, 2011 at 1:21 PM, Dirk Pranke <dpranke at chromium.org> wrote:
>>>>> On Tue, Jul 12, 2011 at 10:01 PM, Dirk Pranke <dpranke at chromium.org> wrote:
>>>>>> On Tue, Jul 12, 2011 at 9:01 PM, Adam Barth <abarth at webkit.org> wrote:
>>>>>>> On Tue, Jul 12, 2011 at 8:34 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:
>>>>>>>> On Tue, Jul 12, 2011 at 8:19 PM, Dirk Pranke <dpranke at chromium.org> wrote:
>>>>>>>>>
>>>>>>>>> Hum. I take it back ... it still wouldn't be a tree, since
>>>>>>>>> chromium-mac-leopard would fall back to chromium-mac-snowleopard, then
>>>>>>>>> mac-leopard, but chromium-mac-snow-leopard would fall back to
>>>>>>>>> mac-snowleopard (giving chromium-mac-snowleopard two parents). And it
>>>>>>>>> looks like chromium-mac-leopard picks up 3,494 baselines from
>>>>>>>>> mac-leopard :(.
>>>>>>>>
>>>>>>>> Can we create chromium-mac and move everything that's shared between
>>>>>>>> chromium-mac-leopard and chromium-mac-snowleopard there?
>>>>>>>> It seems wrong for chromium-mac-leopard to fallback to
>>>>>>>> chromium-mac-snowleopard.
>>>>>>>
>>>>>>> This somewhat surprising fallback strategy is common across ports.
>>>>>>> The "why" is explained on this wiki page:
>>>>>>>
>>>>>>> http://trac.webkit.org/wiki/LayoutTestsSearchPath
>>>>>>>
>>>>>>
>>>>>> In addition, we do actually have a 'chromium-mac'; we don't have a
>>>>>> 'chromium-mac-snowleopard'. I think I mixed that in my mind while
>>>>>> typing this with the apple mac ports, where there are mac-leopard,
>>>>>> mac-sl, and mac ports (the latter representing lion/future).
>>>>>>
>>>>>> Once Lion ships, chromium will undoubtedly add a chromium-mac-snowleopard dir.
>>>>>>
>>>>>> -- Dirk
>>>>>>
>>>>>
>>>>> Okay, I pulled together a slightly more comprehensive report ... in
>>>>> short, we pull things from everywhere. Maybe this is useful to someone
>>>>> if they want to try and treeify the fallbacks :)
>>>>>
>>>>> The format should be fairly self-explanatory. It is a rollup report
>>>>> for all of the baselines, grouped on the combination of ports,
>>>>> platforms, and type of baselines. The first column is the
>>>>> port/platform configuration. The second is the location of the test
>>>>> ("generic" means not in a platform/* directory). The third is the type
>>>>> of baseline for the test, the fourth is the location of the baseline
>>>>> used, and the fifth is the total # of such baselines in that location.
>>>>
>>>> To confirm my understanding:
>>>>
>>>> This row means that the Chromium Mac port running on Snow Leopard gets
>>>> at least 5567 -expected.png files from the LayoutTests/platform/mac
>>>> directory?
>>>>
>>>> chromium-mac-snowleopard,generic,png,mac,5567
>>>>
>>>
>>> That is correct.
>>>
>>>> This is great data!  If you're interested in crunching numbers, it
>>>> might be interested to hack up the deduplicate-tests script to figure
>>>> out how much of the possible sharing we're realizing with our current
>>>> fallback graph.
>>>
>>> I'm not sure I follow what you have in mind here ...
>>
>> No worries.  I'll figure it out myself.
>
> There are approximately 1500 redundant test results that we aren't
> able to collapse using our current fallback strategy.
>
> Adam
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>