[webkit-dev] git.webkit.org problem

[Peter Beverloo]

Hi Lucas,

Thank you very much for your answer!

One change that was applied to Chromium's build bot console[1] recently was
to cache the overview page for a minute.  This may be relevant to the
Apache caching layer you mentioned.  If requests are really handled
serially then updating to version 0.8.5 would hopefully yield performance
improvements as well.

Peter

[1] http://chromium-build-master.appspot.com/p/chromium/console

On Wed, Feb 29, 2012 at 19:32, Lucas Forschler <lforschler at apple.com> wrote:

> Hi Peter,
>
> I am looking into improving performance of build.webkit.org.  After doing
> some initial research I feel that buildbot isn't the most efficient in
> terms of processing a large number of requests.  I believe requests are
> processed serially, and not in parallel.  The buildbot documentation
> mentions a limit to the number of slaves that can be supported, but doesn't
> give any specific numbers.  As webkit has grown, and the number of builds
> and slaves we are supporting has increased, it seems the server is showing
> signs of reaching this capacity.  We may have reached a limit on the number
> of slaves we can handle with our current configuration.
>
> Some of the things I have been researching to increase build.webkit.orgperformance include the following:
>
> -Upgrades to hardware.  It's possible faster hardware could alleviate some
> problems.  It is unclear if this is really the limiting factor, or how much
> additional capacity this would provide.
> -Updating the way files are transferred
> (https://bugs.webkit.org/show_bug.cgi?id=73484)
> -Upgrade the version of buildbot. (0.8.3 -> 0.8.5).
> Version 0.8.4 and higher supposedly has a threading improvement, but it's
> unclear how much it would benefit us since it looks to be mostly database
> related.
> -Upgrading the version of twisted running on the sever.  We are currently
> at version 8.2, and version 12 is now available.
> -Investigating some kind of apache caching layer on top of buildbot
> -Adding a second master to the buildbot system.
>  buildbot supports multi-master mode.  We could potentially split out
> build and test onto two separate masters.
> build.webkit.org and test.webkit.org ?
>
> I would be interested in hearing any other thoughts on increasing
> build.webkit.org performance.
>
> Lucas
>
>
> On Feb 29, 2012, at 7:08 AM, Peter Beverloo wrote:
>
> Hi Bill,
>
> Would it be possible to get some more information on your/Lucas' plans on
> improving stability of the infrastructure?  This was the third significant
> interruption within a week, and especially for us non-PST folks it's
> becoming quite inconvenient.
>
> Thanks,
> Peter
>
> On Wed, Feb 29, 2012 at 15:00, William Siegrist <wsiegrist at apple.com>wrote:
>
>> This should be fixed now.
>>
>> -Bill
>>
>> On Feb 29, 2012, at 3:19 AM, David Barr wrote:
>>
>> >> Is it any trick to update my git-svn repo from svn?
>> >>
>> >> I set up my git svn repo about
>> >> $http://trac.webkit.org/wiki/UsingGitWithWebKit :
>> >> - git clone git://git.webkit.org/WebKit.git WebKit
>> >> - git svn init --prefix=origin/ -T trunk
>> >> http://svn.webkit.org/repository/webkit
>> >> - git config --replace svn-remote.svn.fetch
>> trunk:refs/remotes/origin/master
>> >
>> > Usually, the only missing step here is 'git svn fetch'.
>> > However, when I run that now I get:
>> >
>> > RA layer request failed: OPTIONS of
>> > 'http://svn.webkit.org/repository/webkit': could not connect to server
>> > (http://svn.webkit.org) at /Users/davidbarr/libexec/git-core/git-svn
>> > line 2139
>> >
>> > --
>> > David Barr.
>> > _______________________________________________
>> > webkit-dev mailing list
>> > webkit-dev at lists.webkit.org
>> > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120229/a788bd70/attachment.html>