[webkit-dev] [webkit meeting notes] build systems

[Kevin Ollivier]

Hi Evan,

On Apr 14, 2010, at 5:16 PM, Evan Martin wrote:

> On Tuesday some people gathered to discuss the build-system bingo
> within WebKit and what we can do to reduce (if not unify) the number
> of project files you need to modify when you rename a file.  I took
> the stage as a representative of Chromium's gyp project (I am
> responsible for its Linux Makefile generator).
> 
> Here's a quick list of the different ports I'm aware of and their
> build systems.  (Sorry for the non-monospaced tabbing, hope it comes
> out ok...)
> 
> mac        xcode
> win          vcproj
> qt            qmake (cross-platform output)
> gtk          autotools
> android    Android.mk  (I heard rumors Android isn't actually upstream
> yet, though (?))
> chrome    gyp  (cross-platform output)
> haiku       jam (?)
> brew        waf (?)

wx uses waf, not sure about brew.

> derivative builds:
> rim          hacked version of qt (qmake)
> efl           qmake
> sony        autotools (like gtk)
> wincairo   win + vsprops
> wince       like wincairo (different platforms: use a tree of vcproj)
> air           hacked win (?)
> 
> Nobody is happy with the current situation where you must edit seven
> files to avoid breaking someone's build; the question is what concrete
> steps can improve it.  Everyone present seemed at least mildly
> positive on migrating to other systems, though everyone is also
> understandably skeptical of how much work would be involved.

[snip]

> Concretely, I'd propose:
> - strongly encourage newer ports to hook on to an existing build,
> since new build systems add overhead for everyone
> - chromium/gyp developers should prove gyp by making it build a
> non-chromium port; that includes subtasks of
>  - getting the gyp+jsc build working again (it has worked in the past)
>  - see how close to the existing mac/win build files we can get with gyp
> 
> In particular, if we can generate build files like the Apple Windows
> WebKit port, there are other projects like the Windows Cairo build and
> the WinCE work that could hopefully benefit with minor tweaking.
> 
>> From there, it would probably fall on other builds (qmake, autotools)
> to see if they can benefit from gyp's shared file lists.  From my
> brief tinkering with the GTK build I think I could generate
> Makefile.am's pretty easily; I lack experience with qmake.  But I also
> think it would be a good thing to simply reduce the number of build
> systems

FWIW, for the wx build, I have waf calculate a list of sources to build by passing in a set of directories (also used for setting includes) and rules about feature defines and port info. So a sample rule list might be "compile any source file in any of the following root directories, and also compile any source file in any wx and curl subdirectory, along with any source file in a cf subdirectory when building on Mac." Thanks to the highly structured nature of the WebKit build system, this works surprisingly well. Compiling a source file list (of even 1200+ files) actually adds almost no time to the build but has saved me many hours of patching build file lists and/or sharing that maintenance work with some other port. 90% of the build fixes I have to commit these days are new platform methods that need a port implementation (or at least a stub).

Since waf is really just a Python API, this is all done in straight Python, and outside of the addition of new directories, I almost never need to update the build system, which is probably one reason most contributors hardly ever touch wx or notice it anymore, even though we're still out there chugging along at filling the implementation holes. ;-) 

Anyway, my $0.02 is that, in terms of immediate bang for the buck, we're probably better off trying to synchronize the build systems automatically in some way. My perception is that Qt developers will want to use qmake, GTK will want to use autotools, etc. and while some build systems could certainly be merged (e.g. as you say the Win and Mac projects for WebKit itself could be gyp-generated), I don't think we'll ever really narrow it down to one master system for a variety of reasons. I do, however, think we may be able to narrow the 'build system updating' process down to one step / one script that updates all ports simultaneously, and do so without too much effort. Since most build systems out there have their data stored in either XML or plain text, converting a build file list from one build system's data format to another is probably not more than a few hours of Python hacking, if that. I even long ago made a start on such an approach in WebKitTools/Scripts/update-sources-list.py, that used the old Bakefile XML to grab the list of common WebCore sources, but I bet it could be updated to pull from the .gyp file lists in no time. 

I really think someone should seriously consider investing some time and resources into improving this issue though, updating a build system doesn't take long but updating 7-10 build systems on almost a daily basis probably adds up to some pretty significant amount of time and energy that could probably be spent on better things.

Regards,

Kevin

> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev