[webkit-dev] bugs.webkit.org and trac.webkit org down?

[Mark Rowe]

On 2012-08-09, at 03:14, Peter Beverloo <peter at chromium.org> wrote:

> On Thu, Aug 9, 2012 at 11:09 AM, Mark Rowe <mrowe at apple.com> wrote:
> On 2012-08-09, at 02:41, Osztrogonac Csaba <oszi at inf.u-szeged.hu> wrote:
> 
> > Hi,
> >
> > bugs.webkit.org and trac.webkit.org is unavailable again. :(
> > Could you check it, please?
> 
> This is caused by a problem on a host that I don't have sufficient privileges on to be able to address the issue myself. I've pinged people that should be able to resolve the issue, but given that it's currently 3am in California it may be a few hours before they're awake and fixing it.
> 
> Thank you. Is there any way an on-call or monitoring system could be set up? While it fortunately occurs much less frequently nowadays than it did earlier in the year, it --please excuse my directness-- is unacceptable for a project the size of WebKit to have critical infrastructure unavailable like this for several hours. Even though it's 3am in California, there are many contributors in Asia and Europe who are severely impacted by this.

We have people online virtually 24x7 that are capable of investigating and addressing issues with the webkit.org infrastructure. I'm one such person. This particular case is an unfortunate combination of events: a configuration error on a subset of the new hardware that webkit.org was recently migrated to has unintentionally limited the number of people that can access the host that is currently experiencing problems, and the person that such issues are escalated to is currently on vacation. It's an unfortunate combination, but also one that is unlikely to repeat.

One thing we should look in to is improving the process for reporting issues with webkit.org infrastructure. webkit-dev isn't an ideal way to report issues that our monitoring system hasn't noticed as there's no separation between the regular discussion and more urgent issues, so the emails can be overlooked.

- Mark

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120809/57c9afbf/attachment.html>