[webkit-dev] git.webkit.org problem

[Ryosuke Niwa]

On Wed, Feb 29, 2012 at 11:32 AM, Lucas Forschler <lforschler at apple.com>wrote:

> I am looking into improving performance of build.webkit.org.  After doing
> some initial research I feel that buildbot isn't the most efficient in
> terms of processing a large number of requests.  I believe requests are
> processed serially, and not in parallel.  The buildbot documentation
> mentions a limit to the number of slaves that can be supported, but doesn't
> give any specific numbers.  As webkit has grown, and the number of builds
> and slaves we are supporting has increased, it seems the server is showing
> signs of reaching this capacity.  We may have reached a limit on the number
> of slaves we can handle with our current configuration.
>

On Chromium side, buildbot appears to leak a great amount of memory. e.g.
we end up running out of memory at some point because buildbot keeps
leaking some memory over time :(  Do you see the same issue on your
buildbot?

-Upgrades to hardware.  It's possible faster hardware could alleviate some
> problems.  It is unclear if this is really the limiting factor, or how much
> additional capacity this would provide.
>

My understanding is that we use a really fast server but still experience a
similar problem.


> -Updating the way files are transferred
> (https://bugs.webkit.org/show_bug.cgi?id=73484)
>

Would it make sense to have a separate apache server, etc... for receiving
these files? twistd doesn't appear to be a cut out for
uploading/downloading megabytes of data.

-Upgrade the version of buildbot. (0.8.3 -> 0.8.5).
> Version 0.8.4 and higher supposedly has a threading improvement, but it's
> unclear how much it would benefit us since it looks to be mostly database
> related.
>

I've been told that upgrading buildbot didn't improve the performance. But
upgrading it to 0.8.5 is probably a good nonetheless since it fixes various
bugs I've encountered while maintaining Chromium bots and also when I ran
buildbot locally.

-Investigating some kind of apache caching layer on top of buildbot
>

MUST DO THIS. This will greatly reduce the latency for most people.

-Adding a second master to the buildbot system.
> buildbot supports multi-master mode.  We could potentially split out build
> and test onto two separate masters.
>  build.webkit.org and test.webkit.org ?
>

build.chromium.org takes this approach. But then we won't be able to see
both build and test step at once though... Would it make more sense to
split per port? e.g. Apple Mac & Apple Win, Qt & Gtk, Chromium & misc or
something along that line?

- Ryosuke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120229/fddea788/attachment.html>