[webkit-dev] git.webkit.org problem

[Lucas Forschler]

I'm currenlty upgrading build.webkit.org from 0.8.3 to 0.8.5.  It should be back online shortly.  
Lucas

On Feb 29, 2012, at 12:38 PM, Ryosuke Niwa wrote:

> On Wed, Feb 29, 2012 at 11:32 AM, Lucas Forschler <lforschler at apple.com> wrote:
> I am looking into improving performance of build.webkit.org.  After doing some initial research I feel that buildbot isn't the most efficient in terms of processing a large number of requests.  I believe requests are processed serially, and not in parallel.  The buildbot documentation mentions a limit to the number of slaves that can be supported, but doesn't give any specific numbers.  As webkit has grown, and the number of builds and slaves we are supporting has increased, it seems the server is showing signs of reaching this capacity.  We may have reached a limit on the number of slaves we can handle with our current configuration.
> 
> On Chromium side, buildbot appears to leak a great amount of memory. e.g. we end up running out of memory at some point because buildbot keeps leaking some memory over time :(  Do you see the same issue on your buildbot?
> 
> -Upgrades to hardware.  It's possible faster hardware could alleviate some problems.  It is unclear if this is really the limiting factor, or how much additional capacity this would provide.
> 
> My understanding is that we use a really fast server but still experience a similar problem.
>  
> -Updating the way files are transferred
> 	(https://bugs.webkit.org/show_bug.cgi?id=73484)
> 
> Would it make sense to have a separate apache server, etc... for receiving these files? twistd doesn't appear to be a cut out for uploading/downloading megabytes of data.
> 
> -Upgrade the version of buildbot. (0.8.3 -> 0.8.5).  
> 	Version 0.8.4 and higher supposedly has a threading improvement, but it's unclear how much it would benefit us since it looks to be mostly database related.
> 
> I've been told that upgrading buildbot didn't improve the performance. But upgrading it to 0.8.5 is probably a good nonetheless since it fixes various bugs I've encountered while maintaining Chromium bots and also when I ran buildbot locally.
> 
> -Investigating some kind of apache caching layer on top of buildbot
> 
> MUST DO THIS. This will greatly reduce the latency for most people.
> 
> -Adding a second master to the buildbot system.
> 	buildbot supports multi-master mode.  We could potentially split out build and test onto two separate masters.
> 	build.webkit.org and test.webkit.org ?
> 
> build.chromium.org takes this approach. But then we won't be able to see both build and test step at once though... Would it make more sense to split per port? e.g. Apple Mac & Apple Win, Qt & Gtk, Chromium & misc or something along that line?
> 
> - Ryosuke
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120229/95f090eb/attachment.html>