[webkit-dev] Build File Maintenance (was Re: Please welcome GYP to the our dysfunctional build family)

Fri Jul 10 11:06:44 PDT 2009

> So, does anyone think this would be a bad idea, or have any
> alternate suggestions on how to improve things?

What about adding support for waf to gyp?

> The only format I'm not sure if we could automate > reliably would be the XCode format (at least, on non-Mac
> machines), because IIUC we'd need some sort of parser for it,
> but Apple is the only port maintaining those directly IIUC,
> as now Chromium will be using GYP to update their XCode 
> projects.

If you hand-edit Xcode project files enough times you start to understand them, but you also may go insane in the process.  I don't know that you have to write a full parser for it, but there is some non-trivial, minimal structure you have to understand to update the file properly.

> So even if we couldn't solve the XCode issue, that
> would drop it to updating two locations tops.

I count 6 build systems in use currently (SCons support was added and removed within the last year):

- Apple's Xcode
- Apple's vcproj (also used by at least one other Windows port)
- wx Bakefile (which will be replaced by waf soon)
- Qt Qmake
- GTK GNUMakefile
- Google's gyp (added recently)

Are any of the other ports going to switch to generating their build files using gyp?

Dave

----- Original Message ----
> From: Kevin Ollivier <kevino at theolliviers.com>
> To: Dimitri Glazkov <dglazkov at chromium.org>
> Cc: Mark Mentovai <mark at chromium.org>; WebKit Development <webkit-dev at lists.webkit.org>
> Sent: Friday, July 10, 2009 8:52:57 AM
> Subject: [webkit-dev] Build File Maintenance (was Re: Please welcome GYP to the our dysfunctional build family)
> 
> Hi Dimitri and all,
> 
> Congrats on getting this into WebKit! Actually, I'm in the middle of a build 
> system switch as well - to waf, a re-write of scons that removed many of the 
> performance issues related to searching and calculating dependencies, and which 
> has added some nice features as well (such as .app bundle building). I haven't 
> completed the switch so I can't do preliminary benchmarks, but I'm pretty sure 
> it's actually as fast or faster than make on *nix/Mac. (And BTW, it will 
> probably make Apple devs happy to hear that I'm no longer using the horrid 
> build-wxwebkit bash script to manage the build, but instead have integrated 
> everything into build-webkit finally!)
> 
> The main reason I bring this up, though, is because I think this sort of thing 
> shows that we're unlikely to centralize our build systems any time soon, and I 
> feel a bit sorry for the core devs who, as they accept new ports, are probably 
> finding it more and more tedious, if not difficult, to make sure all the 
> projects get updated by a change to the common parts of the build. They've been 
> very helpful in terms of trying to keep the ports in shape when they make 
> changes, and I feel like I'd like to do what I can to make it easier and faster 
> to keep the other ports in sync.
> 
> At one point, I started on a script (located in 
> WebKitTools/Scripts/update-sources-list.py) whose idea was to take the list of 
> common sources from one file, and make changes to MSVC, Qt, GTK, etc. build 
> systems, so that WebKit devs need only add the file once, run the script, and 
> commit the results. I got it as far as theoretically generating the Qt and GTK 
> file lists, but I don't think anyone on those ports tried integrating it into 
> their build system, and I sort of moved on to other things.
> 
> Unfortunately, right now I'm really swamped (my build system rewrite was 
> prompted by WebKit exceeding internal, hardcoded, Bakefile limits, not by 
> choice), but if a common location for the source files list could be decided 
> upon, I really think a script would be a simple matter to write up (even in Perl 
> :P ), and I think it would probably save developers time and reduce build 
> breakages as well, which I think would add up to a lot of saved time for a lot 
> of people over the long term. The only format I'm not sure if we could automate 
> reliably would be the XCode format (at least, on non-Mac machines), because IIUC 
> we'd need some sort of parser for it, but Apple is the only port maintaining 
> those directly IIUC, as now Chromium will be using GYP to update their XCode 
> projects. So even if we couldn't solve the XCode issue, that would drop it to 
> updating two locations tops.
> 
> So, does anyone think this would be a bad idea, or have any alternate 
> suggestions on how to improve things? If not, is anyone willing to take the ball 
> and run with it? I'd be willing to invest some more time in it, but my ability 
> to commit to it over the next couple weeks at least would be very limited.
> 
> Thanks,
> 
> Kevin
> 
> On Jul 9, 2009, at 9:23 PM, Dimitri Glazkov wrote:
> 
> > Dear WebKiteurs,
> > 
> > In our persisting quest to be more like a common WebKit port, we have
> > added Chromium build files to the tree this afternoon. These files are
> > WebCore/WebCore.gypi and JavaScriptCore/JavaScriptCore.gypi and they
> > are the GYP include files. As you may know, we use GYP
> > (http://code.google.com/p/gyp) for generating MSVC, XCode, Scons, and
> > even Make projects for Chromium.
> > 
> > We are rather fond of GYP. Perhaps it is because it allows us to
> > maintain one set of project files for all three Chromium platforms;
> > 
> > or maybe because it lets us to do things like WebCore.gypi, where we
> > can just mindlessly add all project files to the list and then use
> > various neat GYP filtering facilities to narrow them down to sets that
> > are relevant for specific builds;
> > 
> > or maybe because it easifies creating cross-platform and
> > cross-build-system targets, actions, and rules;
> > 
> > or maybe because we just love saying "Gyp!"
> > 
> > I don't truthfully know.
> > 
> > What I do know is that starting now, we'd love for you to remember
> > WebCore.gypi and JavaScriptCore.gypi when you are adding or removing
> > files from WebCore or JavaScriptCore. Thanks to the power of GYP, you
> > don't have worry whether this file will be used by Chromium: the rule
> > is that if there's a project file change, it applies to the *.gypi
> > files. The format is very simple and intuitive, a simple Python/JSONey
> > list+dict.
> > 
> > Thank you for your attention, men and women of WebKit.
> > 
> > :DG<
> > _______________________________________________
> > webkit-dev mailing list
> > webkit-dev at lists.webkit.org
> > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
> 
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev