[webkit-dev] LayoutTests take too long to run

[Maciej Stachowiak]

On Jul 31, 2009, at 10:25 PM, Eric Seidel wrote:

> 681.70s total testing time
>
> That's 11.5 minutes for every patch I want to land.  (because I run  
> the layout tests before landing anything, as part of bugzilla-tool).
>
> I'm very interested in any suggestions folks have to make that  
> number smaller.  I know Chromium runs the layout tests in parallel  
> using their python run_webkit_tests.py harness.  Try-bots would be  
> another solution to the "make sure it passes all the tests before  
> landing".
>
> Again, very interested in suggestions as to how to improve this.

We're working on try-bots for the ports of particular interest to  
Apple. But independent of that I think speeding up the layout tests is  
a worthwhile goal.

Besides parallelism for better performance on multi-core systems, here  
are some other ideas:

- Look at the slowest tests and see if they can be made to run faster  
(I'm running with --slowest as we speak).
- Look at the subdirectories that take the most time - it's possible  
that in some cases we can consolidate multiple tests to make the whole  
test suite run faster, if sheer number of tests is the bottleneck.

I think a good goal to shoot for would be for the full test suite to  
run in under 5 minutes on a reasonably modern laptop. That should  
allow for reasonable use during development with some room for growth.

Regards,
Maciej