[webkit-dev] run-webkit-tests is moving to parallell testing by default (this weekend)
Ojan Vafai
ojan at chromium.org
Mon Dec 5 11:08:39 PST 2011
I looked at one example that didn't exit early:
http://build.webkit.org/builders/SnowLeopard%20Intel%20Release%20%28Tests%29/builds/35153/steps/layout-test/logs/stdio
In that case, the http tests were the long tail and took 6 minutes longer
than all the other tests. We don't split the http tests up because every
time we've tried it's caused too much flakiness. It's unclear if the
flakiness points to a bug in the test harness (e.g. in how we setup apache)
or to bugs in the tests themselves or both. If someone has time to look
into this, this is probably the biggest benefit to be found in NRWT runtime
when running tests in parallel.
FYI, NRWT outputs a log of the runtime after each run:
2011-12-03 03:09:30,018 58036 printing.py:462 INFO worker/9:
4696 tests, 1746.63 secs
2011-12-03 03:09:30,018 58036 printing.py:462 INFO worker/8:
1177 tests, 1693.47 secs
2011-12-03 03:09:30,018 58036 printing.py:462 INFO worker/3:
1408 tests, 2033.51 secs
2011-12-03 03:09:30,018 58036 printing.py:462 INFO worker/2:
941 tests, 2119.65 secs
2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/1:
1121 tests, 2041.97 secs
2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/0:
1453 tests, 2515.75 secs
2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/7:
1189 tests, 1731.12 secs
2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/6:
3556 tests, 2114.37 secs
2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/5:
948 tests, 2097.13 secs
2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/4:
1411 tests, 1716.66 secs
2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/15:
795 tests, 2027.16 secs
2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/14:
1123 tests, 1732.72 secs
2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/13:
425 tests, 2021.25 secs
2011-12-03 03:09:30,019 58036 printing.py:462 INFO worker/12:
1175 tests, 1710.09 secs
2011-12-03 03:09:30,020 58036 printing.py:462 INFO worker/11:
3462 tests, 2096.30 secs
2011-12-03 03:09:30,020 58036 printing.py:462 INFO worker/10:
1449 tests, 1722.68 secs
2011-12-03 03:09:30,020 58036 printing.py:462 INFO 31120.45
cumulative, 1945.03 optimal
That shows you that, if we fully sharded all the tests, they would in
theory take 1945 seconds to run, but worker/0 (the worker that runs the
http tests) took 2515 seconds to run.
Ojan
On Mon, Dec 5, 2011 at 6:55 AM, Adam Roben <aroben at apple.com> wrote:
> On Dec 2, 2011, at 6:55 PM, Eric Seidel wrote:
>
> > The SnowLeopard bot went from a 1 hr 4 min (!?!) cycle time, to 38 min
> (still !?!).
>
> I suspect our Mac test bots could use a dose of RAM. Many of them only
> have 3GB, since when you're running tests one by one you don't really need
> much more.
>
> -Adam
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20111205/94aa11d4/attachment.html>
More information about the webkit-dev
mailing list