[webkit-dev] LayoutTests results fallback graph

[Adam Barth]

That's correct.  Theoretically, there might exist a fallback strategy
that could remove this redundancy, but it might be arbitrarily
complicated.  Although one could argue that our current strategy is
approaching arbitrary complexity.  :)

Adam


On Wed, Jul 13, 2011 at 2:21 PM, Eric Seidel <eric at webkit.org> wrote:
> I believe what Adam means by this (it wasn't immediately clear to me),
> is that we have 1500 redundant result files with duplicate git hashes
> to some other file.  This could be calculated by the
> deduplicate_results.py script by removing any of the current fallback
> logic and just look at raw duplicates.
>
> Please correct me if I've misunderstood.
>
> -eric
>
> On Wed, Jul 13, 2011 at 2:06 PM, Adam Barth <abarth at webkit.org> wrote:
>> On Wed, Jul 13, 2011 at 1:56 PM, Adam Barth <abarth at webkit.org> wrote:
>>> On Wed, Jul 13, 2011 at 1:36 PM, Dirk Pranke <dpranke at chromium.org> wrote:
>>>> On Wed, Jul 13, 2011 at 1:28 PM, Adam Barth <abarth at webkit.org> wrote:
>>>>> On Wed, Jul 13, 2011 at 1:21 PM, Dirk Pranke <dpranke at chromium.org> wrote:
>>>>>> On Tue, Jul 12, 2011 at 10:01 PM, Dirk Pranke <dpranke at chromium.org> wrote:
>>>>>>> On Tue, Jul 12, 2011 at 9:01 PM, Adam Barth <abarth at webkit.org> wrote:
>>>>>>>> On Tue, Jul 12, 2011 at 8:34 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:
>>>>>>>>> On Tue, Jul 12, 2011 at 8:19 PM, Dirk Pranke <dpranke at chromium.org> wrote:
>>>>>>>>>>
>>>>>>>>>> Hum. I take it back ... it still wouldn't be a tree, since
>>>>>>>>>> chromium-mac-leopard would fall back to chromium-mac-snowleopard, then
>>>>>>>>>> mac-leopard, but chromium-mac-snow-leopard would fall back to
>>>>>>>>>> mac-snowleopard (giving chromium-mac-snowleopard two parents). And it
>>>>>>>>>> looks like chromium-mac-leopard picks up 3,494 baselines from
>>>>>>>>>> mac-leopard :(.
>>>>>>>>>
>>>>>>>>> Can we create chromium-mac and move everything that's shared between
>>>>>>>>> chromium-mac-leopard and chromium-mac-snowleopard there?
>>>>>>>>> It seems wrong for chromium-mac-leopard to fallback to
>>>>>>>>> chromium-mac-snowleopard.
>>>>>>>>
>>>>>>>> This somewhat surprising fallback strategy is common across ports.
>>>>>>>> The "why" is explained on this wiki page:
>>>>>>>>
>>>>>>>> http://trac.webkit.org/wiki/LayoutTestsSearchPath
>>>>>>>>
>>>>>>>
>>>>>>> In addition, we do actually have a 'chromium-mac'; we don't have a
>>>>>>> 'chromium-mac-snowleopard'. I think I mixed that in my mind while
>>>>>>> typing this with the apple mac ports, where there are mac-leopard,
>>>>>>> mac-sl, and mac ports (the latter representing lion/future).
>>>>>>>
>>>>>>> Once Lion ships, chromium will undoubtedly add a chromium-mac-snowleopard dir.
>>>>>>>
>>>>>>> -- Dirk
>>>>>>>
>>>>>>
>>>>>> Okay, I pulled together a slightly more comprehensive report ... in
>>>>>> short, we pull things from everywhere. Maybe this is useful to someone
>>>>>> if they want to try and treeify the fallbacks :)
>>>>>>
>>>>>> The format should be fairly self-explanatory. It is a rollup report
>>>>>> for all of the baselines, grouped on the combination of ports,
>>>>>> platforms, and type of baselines. The first column is the
>>>>>> port/platform configuration. The second is the location of the test
>>>>>> ("generic" means not in a platform/* directory). The third is the type
>>>>>> of baseline for the test, the fourth is the location of the baseline
>>>>>> used, and the fifth is the total # of such baselines in that location.
>>>>>
>>>>> To confirm my understanding:
>>>>>
>>>>> This row means that the Chromium Mac port running on Snow Leopard gets
>>>>> at least 5567 -expected.png files from the LayoutTests/platform/mac
>>>>> directory?
>>>>>
>>>>> chromium-mac-snowleopard,generic,png,mac,5567
>>>>>
>>>>
>>>> That is correct.
>>>>
>>>>> This is great data!  If you're interested in crunching numbers, it
>>>>> might be interested to hack up the deduplicate-tests script to figure
>>>>> out how much of the possible sharing we're realizing with our current
>>>>> fallback graph.
>>>>
>>>> I'm not sure I follow what you have in mind here ...
>>>
>>> No worries.  I'll figure it out myself.
>>
>> There are approximately 1500 redundant test results that we aren't
>> able to collapse using our current fallback strategy.
>>
>> Adam
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>
>