[webkit-dev] The tree is on fire: a tragedy of the commons
mjs at apple.com
Fri Feb 26 09:37:10 PST 2010
On Feb 26, 2010, at 1:36 AM, Adam Barth wrote:
> Not to point fingers, but we've been having trouble keeping
> build.webkit.org green these past few weeks. As I write this message,
> every platform is broken, again. As the project scales, polluting the
> build with brokenness impacts more developers and drains more
> Here are some approaches we could use to turn this tragedy of the
> commons around:
> 1) Adopt a "rollout first, ask questions later" ethic. The vast
> majority of changes are not important enough to break the build for
> everyone else. If we adopt a "rollout first, ask questions later"
> ethic, committers would feel free to rollout brokenness to unbreak the
> build and contributors shouldn't be offended if their patch is rolled
> out without their knowledge. We can always re-land the broken patch
> later once it actually works.
> 2) Require pre-commit vetting of patches. We have the resources to
> build and test every patch on at least one platform before landing the
> patch in the main tree. Vetting patches before landing will help us
> avoid breaking every platform at once. Once the patch has been
> vetted, it can either be landed mechanically (i.e., by commit-queue)
> or manually.
> Here's how I would design the life and times of a patch:
> 1) Contributor uploads patch and nominates it for review.
> 2) Patch vetted by the EWS on numerous platforms.
> 3) If the EWS finds a problem, return to step 1.
> 4) Reviewer marks patch review+.
> 5) Committer decides the patch is ready to land.
> 6) Patch built and tested against top-of-tree on at least one
> 7) If the patch fail to build or pass tests, return to step 1.
> 8) Patch landed.
> 9) If the patch turns any of the "core builders" red, patch is rolled
> out, return to step 1.
> I suspect most of our brokenness coming from committers skipping
> steps 6 and 7.
One data point: I broke the build this weekend, because I introduced a
problem that affected debug builds but not release. I did a full
release build on my own system before committing. When someone pointed
out the breakage, I rolled the patch out myself until I could fix it.
If the problems were such that I could fix them as quickly as rolling
out, I would
I feel like the biggest failure in my case was that I forgot to look
at the bot once my patch went through a cycle. This is why I wish it
would do some form of more active notification. Sometimes I get
distracted after committing and forget to keep hitting reload on the
More information about the webkit-dev