[webkit-dev] Build File Maintenance (was Re: Please welcome GYP to the our dysfunctional build family)

[Kevin Ollivier]

Hi Dimitri and all,

Congrats on getting this into WebKit! Actually, I'm in the middle of a  
build system switch as well - to waf, a re-write of scons that removed  
many of the performance issues related to searching and calculating  
dependencies, and which has added some nice features as well (such  
as .app bundle building). I haven't completed the switch so I can't do  
preliminary benchmarks, but I'm pretty sure it's actually as fast or  
faster than make on *nix/Mac. (And BTW, it will probably make Apple  
devs happy to hear that I'm no longer using the horrid build-wxwebkit  
bash script to manage the build, but instead have integrated  
everything into build-webkit finally!)

The main reason I bring this up, though, is because I think this sort  
of thing shows that we're unlikely to centralize our build systems any  
time soon, and I feel a bit sorry for the core devs who, as they  
accept new ports, are probably finding it more and more tedious, if  
not difficult, to make sure all the projects get updated by a change  
to the common parts of the build. They've been very helpful in terms  
of trying to keep the ports in shape when they make changes, and I  
feel like I'd like to do what I can to make it easier and faster to  
keep the other ports in sync.

At one point, I started on a script (located in WebKitTools/Scripts/ 
update-sources-list.py) whose idea was to take the list of common  
sources from one file, and make changes to MSVC, Qt, GTK, etc. build  
systems, so that WebKit devs need only add the file once, run the  
script, and commit the results. I got it as far as theoretically  
generating the Qt and GTK file lists, but I don't think anyone on  
those ports tried integrating it into their build system, and I sort  
of moved on to other things.

Unfortunately, right now I'm really swamped (my build system rewrite  
was prompted by WebKit exceeding internal, hardcoded, Bakefile limits,  
not by choice), but if a common location for the source files list  
could be decided upon, I really think a script would be a simple  
matter to write up (even in Perl :P ), and I think it would probably  
save developers time and reduce build breakages as well, which I think  
would add up to a lot of saved time for a lot of people over the long  
term. The only format I'm not sure if we could automate reliably would  
be the XCode format (at least, on non-Mac machines), because IIUC we'd  
need some sort of parser for it, but Apple is the only port  
maintaining those directly IIUC, as now Chromium will be using GYP to  
update their XCode projects. So even if we couldn't solve the XCode  
issue, that would drop it to updating two locations tops.

So, does anyone think this would be a bad idea, or have any alternate  
suggestions on how to improve things? If not, is anyone willing to  
take the ball and run with it? I'd be willing to invest some more time  
in it, but my ability to commit to it over the next couple weeks at  
least would be very limited.

Thanks,

Kevin

On Jul 9, 2009, at 9:23 PM, Dimitri Glazkov wrote:

> Dear WebKiteurs,
>
> In our persisting quest to be more like a common WebKit port, we have
> added Chromium build files to the tree this afternoon. These files are
> WebCore/WebCore.gypi and JavaScriptCore/JavaScriptCore.gypi and they
> are the GYP include files. As you may know, we use GYP
> (http://code.google.com/p/gyp) for generating MSVC, XCode, Scons, and
> even Make projects for Chromium.
>
> We are rather fond of GYP. Perhaps it is because it allows us to
> maintain one set of project files for all three Chromium platforms;
>
> or maybe because it lets us to do things like WebCore.gypi, where we
> can just mindlessly add all project files to the list and then use
> various neat GYP filtering facilities to narrow them down to sets that
> are relevant for specific builds;
>
> or maybe because it easifies creating cross-platform and
> cross-build-system targets, actions, and rules;
>
> or maybe because we just love saying "Gyp!"
>
> I don't truthfully know.
>
> What I do know is that starting now, we'd love for you to remember
> WebCore.gypi and JavaScriptCore.gypi when you are adding or removing
> files from WebCore or JavaScriptCore. Thanks to the power of GYP, you
> don't have worry whether this file will be used by Chromium: the rule
> is that if there's a project file change, it applies to the *.gypi
> files. The format is very simple and intuitive, a simple Python/JSONey
> list+dict.
>
> Thank you for your attention, men and women of WebKit.
>
> :DG<
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev