[webkit-dev] On the greeness of the GTK+ bot

Dimitri Glazkov dglazkov at chromium.org
Mon Nov 16 09:47:54 PST 2009

On Mon, Nov 16, 2009 at 5:56 AM, Xan Lopez <xan at gnome.org> wrote:
> On Mon, Nov 16, 2009 at 3:33 PM, Gustavo Noronha Silva <gns at gnome.org> wrote:
>> On Mon, 2009-11-16 at 05:24 -0800, Adam Barth wrote:
>>> Eric and I are working on a bot that might help this situation.
>>> Essentially, the bot will try out patches on Qt and GTK and add a
>>> comment to the bug if the patch regresses the build.  Our plan is to
>>> start with compiling, but we'd eventually like to run the tests as
>>> well.
>> That sounds like an awesome idea. Thanks for working on it. Build
>> breakages themselves have become less of an issue for us in recent times
>> - people are definitely more aware of our bots, and are acting on fixing
>> them when they break.
>> I think such an automated approach to running the build, and tests for
>> upcoming patches will surely help with giving this a second step
>> forward.
> This is nice to see, but as Gustavo says build breakage is not too
> much an issue at the moment for us: the build does not break very
> often, and when it does people usually take the time to figure out
> what happened and either do fix it themselves or poke us about it.
> That being said, this could be improved in any number of ways and I'm
> happy to see it getting ever better.
> What is effectively a black hole with respect to our time is the tests
> breakage, though. We get new tests failing very regularly (either
> through new tests or through new code making old tests fail), and once
> the bots are red people tend to pay even less attention to them, so
> they spiral out of control in a positive feedback loop until we have
> tests failing in the double digits in a matter of days (or hours!). Of
> course in an ideal world we'd have a team big enough to always have at
> least one person looking at this and fixing the problems the moment
> they arise, but unfortunately this is not the case.

This is a huge issue with the Chromium port as well. We spend quite a
bit of effort tracking down failing tests, only to discover that the
failure is due to one-port baselines or new functionality added to
DRT. I wonder if the approach we have today in regards to tests is not
sustainable with multiple vibrant ports, each spending way to much
time catching up.


More information about the webkit-dev mailing list