[Webkit-unassigned] [Bug 84643] Incorporate Chromium buildbot waterfall improvements into the webkit.org waterfall page

Mon Apr 23 15:37:30 PDT 2012

https://bugs.webkit.org/show_bug.cgi?id=84643

Dirk Pranke <dpranke at chromium.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sfalken at apple.com

--- Comment #2 from Dirk Pranke <dpranke at chromium.org>  2012-04-23 15:37:27 PST ---
Also, copying sfalken on this; we didn't get to talk in person, but I think I think I talked to someone (darin?) about this, at least ...

the topic of being able to shard the layout tests across multiple bots came up at the meeting. I've spent a pretty good amount of time thinking about this, and there's an easy way (which we do today for some of the debug bots) and a much harder way.

The easy way is to just manually run parts of the tests as a step on each bot (e.g., split the tests in half, and one run half on one bot and one half on the other). For example:

http://build.chromium.org/p/chromium.webkit/waterfall?builder=Webkit+Win+Builder+(dbg)&builder=Webkit+Win+(dbg)(1)&builder=Webkit+Win+(dbg)(2)

We don't make any attempt to merge the results of the two test bots (except that the flakiness dashboard deals with this somehow). 

Doing some sort of automatic sharding and execution would require us to synchronize the filesystem across the two machines, or do a remote copy of the binary onto the targets plus converting the local-file tests to http tests (either directly or indirectly). It's all quite doable but it would probably take a while to kick out the bugs and make sure it's stable. 

The alternative is to just use bigger iron and spend a little time tweaking the tests to run stably in parallel. On my Mac Pro I can run the tests in 3 minutes with 12 chromium release DRTs with minimal flakiness if I spend a little time stubbing out flaky tests. Get things much faster than that and you start not seeing a real benefit because other steps like update and archive are still slow.

If we think there's still interest in going down this road it deserves its own bug, though ...

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.