[webkit-dev] LayoutTests take too long to run

[Ryosuke Niwa]

For editing/selection/extend-selection.html,
It seems like the bottleneck is testExtendingSelection, in particular,
positionExtendingInDirection and checkSameOrder:

function positionsExtendingInDirection(sel, direction, granularity)
{
    var positions = [];

    while (true) {
        positions.push({ node: sel.extentNode, begin: sel.baseOffset, end:
sel.extentOffset });
        sel.modify("extend", direction, granularity);
        if (positions[positions.length - 1].node == sel.extentNode &&
positions[positions.length - 1].end == sel.extentOffset)
            break;
    };
    return positions;
}

We return positions and compare the results for LTR and RTL
using checkSameOrder

function checkSameOrder(inputPositions, inputSamePositions)
{
    var positions = inputPositions.slice();
    var samePositions = inputSamePositions.slice();
    var mismatch = positions.length != samePositions.length;
    if (mismatch)
        log("WARNING: positions should be the same, but the length are not
the same " +  positions.length + " vs. " + samePositions.length + "\n");
    while (!mismatch) {
        var pos = positions.pop();
        if (!pos)
            break;
        var samePos = samePositions.pop();
        if (pos.node != samePos.node) {
            mismatch = true;
            log("WARNING: positions should be the same, but node are not the
same\n");
        }
        if (pos.begin != samePos.begin) {
            mismatch = true;
            log("WARNING: positions should be the same, but begin are not
the same " + pos.begin + " vs. " + samePos.begin + "\n");
        }
        if (pos.end != samePos.end) {
            mismatch = true;
            log("WARNING: positions should be the same, but end are not the
same " + pos.end + " vs. " + samePos.end + "\n");
        }
   }
}

Can anyone think of any way to make this faster?

Ryosuke

On Sat, Aug 1, 2009 at 9:49 AM, Eric Seidel <eric at webkit.org> wrote:

> Yeah, I stared a slowest run before going to bed, and got the following:
> The 10 slowest tests:
>
> 9.58 secs: editing/selection/move-left-right.html
> 9.37 secs: fast/js/array-filter.html
> 7.71 secs: editing/selection/extend-selection.html
> 6.82 secs: fast/js/sort-randomly.html
> 6.25 secs: http/tests/cache/subresource-expiration.html
> 5.77 secs: fast/js/array-enumerators-functions.html
> 5.41 secs: fast/events/click-count.html
> 5.39 secs: fast/js/toString-and-valueOf-override.html
> 5.26 secs: fast/js/try-catch-crash.html
> 5.05 secs: http/tests/navigation/slowmetaredirect-basic.html
>
> that's 10% of our time right there. :)  I bet some of those have 5 second
> timers and should be easy to fix.  I'll take a look, anyone else who would
> like to help would be most welcome. :)
>
> I remember Ojan finding that running tests in parallel was the biggest
> overall win, but there is definitely a lot of low-hanging fruit in the
> slowest tests too.  I'm curious what the fastest tests run in.  I would
> expect the harness cost to be near-zero, but I'm not sure.  (i.e. what's the
> runtime of run-webkit-tests with 10000 empty test.  I would expect only a
> few seconds.)
>
> -eric
>
> On Fri, Jul 31, 2009 at 11:21 PM, Maciej Stachowiak <mjs at apple.com> wrote:
>
>>
>> On Jul 31, 2009, at 10:25 PM, Eric Seidel wrote:
>>
>>  681.70s total testing time
>>>
>>> That's 11.5 minutes for every patch I want to land.  (because I run the
>>> layout tests before landing anything, as part of bugzilla-tool).
>>>
>>> I'm very interested in any suggestions folks have to make that number
>>> smaller.  I know Chromium runs the layout tests in parallel using their
>>> python run_webkit_tests.py harness.  Try-bots would be another solution to
>>> the "make sure it passes all the tests before landing".
>>>
>>> Again, very interested in suggestions as to how to improve this.
>>>
>>
>> We're working on try-bots for the ports of particular interest to Apple.
>> But independent of that I think speeding up the layout tests is a worthwhile
>> goal.
>>
>> Besides parallelism for better performance on multi-core systems, here are
>> some other ideas:
>>
>> - Look at the slowest tests and see if they can be made to run faster (I'm
>> running with --slowest as we speak).
>> - Look at the subdirectories that take the most time - it's possible that
>> in some cases we can consolidate multiple tests to make the whole test suite
>> run faster, if sheer number of tests is the bottleneck.
>>
>> I think a good goal to shoot for would be for the full test suite to run
>> in under 5 minutes on a reasonably modern laptop. That should allow for
>> reasonable use during development with some room for growth.
>>
>> Regards,
>> Maciej
>>
>>
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20090801/66ca9de3/attachment.html>