[webkit-dev] Bot watching and Apple bots

[Maciej Stachowiak]

On Apr 25, 2012, at 2:34 PM, Adrienne Walker <enne at google.com> wrote:

> Hi-
> 
> This has been on my mind lately too, so I'm happy that you brought it
> up.  I have a couple of questions:
> 
> Would it be possible to make mac's ews warn about test failures in the
> same way that chromium-ews does? That's been super helpful for keeping
> Chromium green.
> 
> Also, when you say "pay attention," what level of involvement should
> we expect for one committer making the bots go red on a port that they
> are not primarily working on? Should we expect test expectation
> rebaselining? Patch reverting? Adding "= FAIL" test expectations and
> filing a bug? A friendly ping in irc? An apology and cupcakes?

Depends on the nature of the breakage. Here's my opinion on a few possible scenarios:

* Compile failure => revert ASAP (unless it can be fixed as fast as a revert)
* Made a significant number of tests crash or hang => revert ASAP (unless it can be fixed as fast as a revert)
* Made tests fail in a way that indicates a real bug => revert or try to fix at your option, but if someone is watching that bot (e.g. Chromium gardeners or Apple bot watchers) let them know what you are up to
* Made tests fail, but only due to need for new improved results; or added new test w/o results for all platforms => update the appropriate results or get help to do so

I think it may help the project overall if we made the buildbot page indicate who's the sheriff/gardner/watcher for any given buildbot. Right now it may be hard to tell who to work with if you caused breakage. Barring actual changes to the buildbot page, another thing we could do is put all this info in a shared wiki page.


We'd also like to improve the infrastructure to make it easier to keep the bots green. Two things we've thought about (though I have nothing to announce yet):

- Find a way to have a Mac EWS that runs tests
- Add more machines to serve the test queues

One thing we've already done:

- Change our testers so that they test the most recent build first, rather than the least recent that's not yet tested, and try to patch holes during downtime

Regards,
Maciej


> 
> I'm just looking for some clarification on what the community
> expectations are with respect to paying attention to the bots of other
> ports.