[webkit-dev] run-webkit-tests is moving to parallell testing by default (this weekend)

[Ojan Vafai]

Why is that? I don't know about other ports, but AFAIK, chromium writes to
a mock clipboard and the Apple mac port writes to a local OS clipboard
instance instead of the global one, specifically to avoid copy/paste tests
interacting. Even without running tests in parallel, it's probably a good
idea in order to avoid copy-pastes the developer is doing from affecting
the test run.

That said, I believe we have a general way of marking subdirectories as
needing to run serially (which is what we do for http) if there are other
reasons we need to.

On Mon, Dec 5, 2011 at 11:12 AM, David Levin <levin at google.com> wrote:

> I believe there are some tests (copy/paste) that it would be very hard to
> fully shard due to how they work.
>
> dave
>
>
>
> On Mon, Dec 5, 2011 at 11:08 AM, Ojan Vafai <ojan at chromium.org> wrote:
>
>> I looked at one example that didn't exit early:
>> http://build.webkit.org/builders/SnowLeopard%20Intel%20Release%20%28Tests%29/builds/35153/steps/layout-test/logs/stdio
>>
>> In that case, the http tests were the long tail and took 6 minutes longer
>> than all the other tests. We don't split the http tests up because every
>> time we've tried it's caused too much flakiness. It's unclear if the
>> flakiness points to a bug in the test harness (e.g. in how we setup apache)
>> or to bugs in the tests themselves or both. If someone has time to look
>> into this, this is probably the biggest benefit to be found in NRWT runtime
>> when running tests in parallel.
>>
>> FYI, NRWT outputs a log of the runtime after each run:
>>
>> 2011-12-03 03:09:30,018 58036 printing.py:462 INFO       worker/9:  4696 tests, 1746.63 secs
>> 2011-12-03 03:09:30,018 58036 printing.py:462 INFO       worker/8:  1177 tests, 1693.47 secs
>> 2011-12-03 03:09:30,018 58036 printing.py:462 INFO       worker/3:  1408 tests, 2033.51 secs
>> 2011-12-03 03:09:30,018 58036 printing.py:462 INFO       worker/2:   941 tests, 2119.65 secs
>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO       worker/1:  1121 tests, 2041.97 secs
>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO       worker/0:  1453 tests, 2515.75 secs
>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO       worker/7:  1189 tests, 1731.12 secs
>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO       worker/6:  3556 tests, 2114.37 secs
>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO       worker/5:   948 tests, 2097.13 secs
>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO       worker/4:  1411 tests, 1716.66 secs
>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO      worker/15:   795 tests, 2027.16 secs
>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO      worker/14:  1123 tests, 1732.72 secs
>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO      worker/13:   425 tests, 2021.25 secs
>> 2011-12-03 03:09:30,019 58036 printing.py:462 INFO      worker/12:  1175 tests, 1710.09 secs
>> 2011-12-03 03:09:30,020 58036 printing.py:462 INFO      worker/11:  3462 tests, 2096.30 secs
>> 2011-12-03 03:09:30,020 58036 printing.py:462 INFO      worker/10:  1449 tests, 1722.68 secs
>> 2011-12-03 03:09:30,020 58036 printing.py:462 INFO    31120.45 cumulative, 1945.03 optimal
>>
>> That shows you that, if we fully sharded all the tests, they would in
>> theory take 1945 seconds to run, but worker/0 (the worker that runs the
>> http tests) took 2515 seconds to run.
>>
>> Ojan
>>
>> On Mon, Dec 5, 2011 at 6:55 AM, Adam Roben <aroben at apple.com> wrote:
>>
>>> On Dec 2, 2011, at 6:55 PM, Eric Seidel wrote:
>>>
>>> > The SnowLeopard bot went from a 1 hr 4 min (!?!) cycle time, to 38 min
>>> (still !?!).
>>>
>>> I suspect our Mac test bots could use a dose of RAM. Many of them only
>>> have 3GB, since when you're running tests one by one you don't really need
>>> much more.
>>>
>>> -Adam
>>>
>>> _______________________________________________
>>> webkit-dev mailing list
>>> webkit-dev at lists.webkit.org
>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>>
>>
>>
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20111205/0c790fbb/attachment.html>