[Webkit-unassigned] [Bug 89391] Would like a way to force a clean build

Thu Jul 19 11:53:05 PDT 2012

https://bugs.webkit.org/show_bug.cgi?id=89391

--- Comment #13 from Josh Hawn <jhawn at apple.com>  2012-07-19 11:53:04 PST ---
Okay, I've got a new plan!

Forget all of the above changes to Buildbot.

The way that a Builder currently chooses which build to start is by picking a slavebuilder and build requests from a list of available slaves and a list of build requests.

First, it chooses which slavebuilder to use. Users can currently customize how they want to choose builders by specifying a 'nextSlave' function in their configuration for a builder. It defaults to using random.choice to pick any of the available slaves.

Then (only after choosing the slave) it chooses which build request to use. Users can also customize which build request is chosen by specifying a 'nextBuild' function in their configuration for a builder. It defaults to using the earliest submitted request.

Finally, (after choosing the slave and build request) it tries to merge build requests to submit for one build. Users can even customize this by specifying a 'mergeRequests' function in their configuration for a builder! It defaults to merging builds with compatible source stamps (check the code/documentation for the details).

The issue here is that we would like to be able to do all three of these things at once: choose a slave, build request, and merge compatible requests.

When we have a build request to clean a slave, we need to choose the request first so that we know which slave to choose (if available). The current implementation doesn't allow that (it chooses build requests independently from choosing the slave, and only after choosing the slave).

So, I propose a simple (much simpler than my last suggestion) change to Buildbot Builder configurations. A configuration option called "chooseSlaveAndMergeBuildRequests". It is the combination of the currently available options: "nextSlave", "nextBuild", and "mergeRequests".

The "chooseSlaveAndMergeBuildRequests" function will take two arguments: a list of available slavebuilders and a list of buildrequest objects. It will return a tuple of (selected_slavebuilder, [buildrequest, ...]) which is a single chosen slavebuilder and a list of build requests to be merged. You can still implement the logic of your "nextSlave", "nextBuild" and "mergeRequests" configuration inside this function, but now it gives you more control to choose build requests that depend on a particular slavebuilder.

This will allow us to inspect the properties of a build request to see if it was scheduled by a certain 'clean' scheduler! Another property set by the scheduler will specify the 'clean_slavename', and we can choose that slave if it is available. We can also merge 'clean' requests for the same buildslave.

Now, about getting that "Clean" button on the builder page...

The clean button can be implemented as a simple ForceScheduler that sets a property for which slave to clean.
Normally, ForceScheduler buttons are listed separately on the page, but we want them listed in the "Buildslaves" section of the page.
We've always been able to customize templates for pages, so we just need to customize the "builder.html" template to put the form for the clean scheduler in the right place. Customizing the "forms.html" template is also possible, so we can make a macro for "clean_scheduler(url, scheduler, slavename)" that just places a button on the page that submits a form to force a build with the 'clean_slavename' property set to the given slavename.

Oh, no, but we still need a different set of steps for this "clean" build!

That can be easily taken care of in a user's configuration as well! 

Each 'BuildFactory' instance has a method called 'newBuild' which takes a list of BuildRequest objects and returns a new 'Build' object that has all of the steps needed to complete the build.
Users can override this method in a subclass of 'BuildFactory' and check the properties of the BuildRequest objects to see if they came from the 'clean' scheduler. It can then return a Build object that has been instantiated with a different set of 'clean steps' instead of the normal build steps. A user would have to specify these 'clean steps' in the configuration.

I've got all of these changes applied on a test master I'm running, and it provides all of the features of my last proposed change, but with minimal changes to Buildbot itself and most changes done in the master.cfg file.

Now there's no need to add new columns to database tables, or have separate factories for each Builder! The "chooseSlaveAndMergeBuildRequests" builder config option allows for more control, and everything else is done in master.cfg and custom templates!

Let me know what you guys think.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.