[webkit-dev] The tree is on fire: a tragedy of the commons

[Ojan Vafai]

On Fri, Feb 26, 2010 at 10:40 AM, Geoffrey Garen <ggaren at apple.com> wrote:

> A lot of the proposals on this thread would interfere with this work flow:
>
> 1. Finish patch and get it working on local machine.
> 2. Check in, automatically test for compatibility on other machines and
> OS's in parallel, resolving unexpected problems as they arise.
>

There is a non-trivial cost of this workflow on the rest of the team.
-keeps the commit-queue from running
-often results in new test failures going unnoticed because the tree is
already red
-we can't generally trust that all the tests should pass locally

Clearly, every developer having access to every environment and knowing how
to setup/build/test on each environment is not an option.

Would it be enough for you if you could send a patch to the EWS and get back
the results for any test failures? This would let you maintain the above
workflow without actually committing. Adam/Eric, how close is the EWS to
enabling that? The missing pieces as I see it are:

1. Running the layout tests as part of the EWS.
2. Giving access to the results of any failing tests.

and change it to this work flow:
>
> 0. Purchase and set up about 15 different build environments.
> 1. Finish patch and get it working on local machine.
> 2. Manually test on build environments purchased and set up in (0).
> 3. Check in.
>
> That would be a serious blow to productivity -- probably a cure that is
> worse than the disease.
>
> Bear in mind that the build environments problem is multiplied by Google's
> choice to use a separate JavaScript engine, which effectively almost doubles
> the testing surface area.
>
> Geoff
>
> On Feb 26, 2010, at 9:44 AM, Eric Seidel wrote:
>
> > The bots take 15 minutes to cycle.  The moment the build is broken,
> > thats FIX_TIME + BOT_CYCLE_TIME until their green again.
> >
> > I think we should cap the fix grace period at something like 15
> > minutes, that means no more than 30 minutes of tree redness per break.
> > That might be too aggressive to start with for WebKit, but I think we
> > should move towards that.
> >
> > I would re-write rule one as something like this:
> > 1.  Comment in the bugzilla bug when the build breaks.  If there is no
> > bugzilla bug, comment in #webkit.
> > 2.  15 minutes after the break or 10 minutes after the comment, with
> > no reply from the breaker, roll out the patch.
> >
> > -eric
> >
> > On Fri, Feb 26, 2010 at 9:32 AM, Nikolas Zimmermann
> > <zimmermann at physik.rwth-aachen.de> wrote:
> >>
> >> Am 26.02.2010 um 18:17 schrieb Dimitri Glazkov:
> >>
> >>> To summarize the thread:
> >>>
> >>> 1) We're adopting "when in doubt, roll it out" approach to patches
> >>> that turn tree red.
> >>> 2) Need to find a way to run Mac-EWS for non-committers.
> >>> 3) Enable "build-break" emails to webkit-dev or another opt-in mailing
> >>> list
> >>>
> >>> What else?
> >>
> >> I'm a bit scared of rule 1. How about we define a minimum delay when to
> >> roll-out patches, after they break something?
> >> Let's say, if a commit breaks the tree, give the commiter a time frame
> of 30
> >> minutes to fix it - otherwhise roll-out (we could even automate that.)
> >>
> >> Example: When landing a SVG patch, that worked fine on Leopard, but
> broke
> >> Snow Leopard, I'd like to have some time to identify wheter it's the
> >> fault of my patch, or a platform specific problem. If it's the fault of
> my
> >> patch, I have no problem with reverting. But if I can't immediately fix
> the
> >> problem, because it's a platform specific issue, which can not be fixed
> (in
> >> terms of WebKit), then adding to the Skipped list, and filing a new bug
> >> just takes 5 minutes. Reverting the whole patch, just to reland it with
> a
> >> Skipped list addition is a bit too much work for me.
> >>
> >> What do others think?
> >>
> >> Cheers,
> >> Niko
> >>
> >> _______________________________________________
> >> webkit-dev mailing list
> >> webkit-dev at lists.webkit.org
> >> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
> >>
> > _______________________________________________
> > webkit-dev mailing list
> > webkit-dev at lists.webkit.org
> > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20100226/43139736/attachment.html>