[webkit-dev] Hanging builds

[Mark Rowe]

On 2011-11-29, at 10:00, Adam Roben wrote:

> On Nov 28, 2011, at 6:35 PM, Adam Roben wrote:
> 
>> On Nov 26, 2011, at 3:34 AM, Nikolas Zimmermann wrote:
>> 
>>> Good morning WebKit folks,
>>> 
>>> I'm looking at build.webkit.org/waterfall, and see several bots which are idle, but still report "building < 1min" pretending they would do something.
>>> Does any of our build.webkit.org masters know the cause of this? They all seem to hang in the "upload" step. Can we detect these errors, and avoid that?
>>> Maybe sth. like a 30min timeout timer for this step?
>>> 
>>> Thanks in advance,
>>> Niko
>>> 
>>> P.S. I'm aware that a master restart would "fix this", but maybe we can try better and avoid that completely.
>> 
>> More and more of these seemed to be piling up, so I restarted the master. I don't know what causes them, though I suspect it has something to do with slaves restarting during a download/upload step.
> 
> It looks like Buildbot doesn't have support for setting a timeout for file transfers. We could write our own file transfer step that uses rsync or similar, which would allow us to use the standard timeout mechanism.

We used rsync for this in the distant past. It's a hassle from the point of view of the build master as it introduces a host of new authentication and permission issues.

- Mark