[webkit-dev] LayoutTests take too long to run

[Ryosuke Niwa]

editing/selection/move-left-right.html has very similar script where we
return positions and compare entires in reverse order.  Since there is no
setTimeout, we need to rewrite those scripts so that they run faster or
improve the performance of WebKit.
Ryosuke

On Sat, Aug 1, 2009 at 1:09 PM, Ryosuke Niwa <rniwa at google.com> wrote:

> For editing/selection/extend-selection.html,
> It seems like the bottleneck is testExtendingSelection, in particular,
> positionExtendingInDirection and checkSameOrder:
>
> function positionsExtendingInDirection(sel, direction, granularity)
> {
>     var positions = [];
>
>     while (true) {
>         positions.push({ node: sel.extentNode, begin: sel.baseOffset, end:
> sel.extentOffset });
>         sel.modify("extend", direction, granularity);
>         if (positions[positions.length - 1].node == sel.extentNode &&
> positions[positions.length - 1].end == sel.extentOffset)
>             break;
>     };
>     return positions;
> }
>
> We return positions and compare the results for LTR and RTL
> using checkSameOrder
>
> function checkSameOrder(inputPositions, inputSamePositions)
> {
>     var positions = inputPositions.slice();
>     var samePositions = inputSamePositions.slice();
>     var mismatch = positions.length != samePositions.length;
>     if (mismatch)
>         log("WARNING: positions should be the same, but the length are not
> the same " +  positions.length + " vs. " + samePositions.length + "\n");
>     while (!mismatch) {
>         var pos = positions.pop();
>         if (!pos)
>             break;
>         var samePos = samePositions.pop();
>         if (pos.node != samePos.node) {
>             mismatch = true;
>             log("WARNING: positions should be the same, but node are not
> the same\n");
>         }
>         if (pos.begin != samePos.begin) {
>             mismatch = true;
>             log("WARNING: positions should be the same, but begin are not
> the same " + pos.begin + " vs. " + samePos.begin + "\n");
>         }
>         if (pos.end != samePos.end) {
>             mismatch = true;
>             log("WARNING: positions should be the same, but end are not the
> same " + pos.end + " vs. " + samePos.end + "\n");
>         }
>    }
> }
>
> Can anyone think of any way to make this faster?
>
> Ryosuke
>
> On Sat, Aug 1, 2009 at 9:49 AM, Eric Seidel <eric at webkit.org> wrote:
>
>> Yeah, I stared a slowest run before going to bed, and got the following:
>> The 10 slowest tests:
>>
>> 9.58 secs: editing/selection/move-left-right.html
>> 9.37 secs: fast/js/array-filter.html
>> 7.71 secs: editing/selection/extend-selection.html
>> 6.82 secs: fast/js/sort-randomly.html
>> 6.25 secs: http/tests/cache/subresource-expiration.html
>> 5.77 secs: fast/js/array-enumerators-functions.html
>> 5.41 secs: fast/events/click-count.html
>> 5.39 secs: fast/js/toString-and-valueOf-override.html
>> 5.26 secs: fast/js/try-catch-crash.html
>> 5.05 secs: http/tests/navigation/slowmetaredirect-basic.html
>>
>> that's 10% of our time right there. :)  I bet some of those have 5 second
>> timers and should be easy to fix.  I'll take a look, anyone else who would
>> like to help would be most welcome. :)
>>
>> I remember Ojan finding that running tests in parallel was the biggest
>> overall win, but there is definitely a lot of low-hanging fruit in the
>> slowest tests too.  I'm curious what the fastest tests run in.  I would
>> expect the harness cost to be near-zero, but I'm not sure.  (i.e. what's the
>> runtime of run-webkit-tests with 10000 empty test.  I would expect only a
>> few seconds.)
>>
>> -eric
>>
>> On Fri, Jul 31, 2009 at 11:21 PM, Maciej Stachowiak <mjs at apple.com>wrote:
>>
>>>
>>> On Jul 31, 2009, at 10:25 PM, Eric Seidel wrote:
>>>
>>>  681.70s total testing time
>>>>
>>>> That's 11.5 minutes for every patch I want to land.  (because I run the
>>>> layout tests before landing anything, as part of bugzilla-tool).
>>>>
>>>> I'm very interested in any suggestions folks have to make that number
>>>> smaller.  I know Chromium runs the layout tests in parallel using their
>>>> python run_webkit_tests.py harness.  Try-bots would be another solution to
>>>> the "make sure it passes all the tests before landing".
>>>>
>>>> Again, very interested in suggestions as to how to improve this.
>>>>
>>>
>>> We're working on try-bots for the ports of particular interest to Apple.
>>> But independent of that I think speeding up the layout tests is a worthwhile
>>> goal.
>>>
>>> Besides parallelism for better performance on multi-core systems, here
>>> are some other ideas:
>>>
>>> - Look at the slowest tests and see if they can be made to run faster
>>> (I'm running with --slowest as we speak).
>>> - Look at the subdirectories that take the most time - it's possible that
>>> in some cases we can consolidate multiple tests to make the whole test suite
>>> run faster, if sheer number of tests is the bottleneck.
>>>
>>> I think a good goal to shoot for would be for the full test suite to run
>>> in under 5 minutes on a reasonably modern laptop. That should allow for
>>> reasonable use during development with some room for growth.
>>>
>>> Regards,
>>> Maciej
>>>
>>>
>>
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20090801/f2a40737/attachment.html>