[webkit-dev] New build system for gdk/linux

[Kevin Ollivier]

Hi Krzysztof,

On Oct 15, 2006, at 4:32 PM, Krzysztof Kowalczyk wrote:

> This e-mail is to let interested people know about my new build system
> for gdk/linux and "sell" it so that hopefully it get integrated into
> official tree.
>
> Why new build system?
>
> I wasn't happy with what bakefiles generate (mostly the unnecessary
> recompile with gneerated makefile and the fact that there's only one
> target and not standard release/debug build). I also plan on trying to
> build a minimal version of gdk/linux (e.g. without xslt or xpath
> support) and I felt that bakefiles won't let me experiment easily.
>
> Why not fix bakefiles?
>
> Bakefiles seem like a good idea but badly implemented and the project
> is almost dead. It seemed faster to do it my way than learn enough
> about bakefiles to fix the things that annoy me.

About the only issue I'm aware of is the DerivedSources issue, which  
happens because Bakefile doesn't have a way to specify a 'true' pre- 
build step using GNU make. To be honest, I ended up moving the  
"DerivedSources" target out of my Unix/Mac makefiles, and instead  
created a build-wxwebkit script that runs regenerates the makefiles,  
runs DerivedSources.make, moves JS headers, etc. and then builds once  
the environment is fully prepped. Such a script is of course  
generally useful as a tool for automating the entire build process,  
so it doesn't seem to me a big issue to do things this way.

Outside of the "pre-build step" issue, though, I've found it works  
very well for me, and I think the problems you're seeing are mostly  
with how the Makefiles, and supporting scripts, have been designed.  
Particularly the 'unnecessary rebuilds', more on that below.

> Why not cmake?
>
> I would love to see cmake-based build for Gdk as well, but I don't  
> know cmake
> and doing it my way seemed simpler and faster (for me) than learning
> about cmake.
>
> What is it?
>
> http://bugs.webkit.org/show_bug.cgi?id=11308
>
> It's a simple python script that generates 2 Makefiles: release and
> debug for gdk/linux, which I call Makefile.gdk.rel and
> Makefile.gdk.dbg. I think it makes sense to keep both the script and
> generated makefiles in the repository to make it easier for people to
> compile gdk/linux version out-of-the-box. Makefiles don't change that
> often so re-generating them is not a big deal.
>
> I kept the nice tricks I've learned from bakefile-based makefiles.
>
> I realize it's unusual to have a makefile per target - usually it's
> done with one makefile with condititional code. One makefile saves dev
> time if a makefile is being written by hand, but it doesn't matter in
> case of a makefile generated by a script. Having separate makefiles
> makes it easier to debug them because more things are spelled out.
>
> What are advantages of this Makefile over bakefiles:
> * builds both debug and release targets, in separate directories;  
> other targets
> easy to add

We can, of course, do this for Bakefile too. Just create an option  
for specifying build target, like make TARGET=debug, and then set the  
build dirs to something like build-$TARGET.

> * no dependency on bakefiles, just python, for generating the  
> makefiles
> * makefile generated by bakefile causes unnecessary rebuilds (i.e.  
> make after
> make should do nothing, but it rebuilds some jsc files)

I did some digging, and the problem (at least, that I see) is not  
Bakefile, it's WebCore/move-js-headers.sh. For more info, see here:

http://bugs.webkit.org/show_bug.cgi?id=11322

The important thing to remember about Bakefile is that it generates  
standard GNU makefiles. So, if you find it re-building targets you  
don't think it should be, you can always run make -d and then grep  
for "Must remake" to find out which targets make thinks are outdated.  
Once you've found that, the lines above it should indicate which  
dependency caused the rebuild to occur. Once you know the dependency,  
you just need to figure out what incorrectly changed it's  
modification time, etc. and stop that from happening. I'm fairly  
confident you can totally remove any unnecessary target re-builds  
using that approach. Bakefiles, like anything else, can have 'bugs'  
in them.

> * one, non-recursive makefile (for some it might be disadvantage;
> supposedly recursive makefile should be considered harmful)

What do you mean by recursive makefiles?

> * meta-description is much shorter (my script uses dir + list of
> exceptions to specify files for the makefile while bakefiles/cmake
> requires listing every file explicitely)
>
> Disadvantages:
> * doesn't build shared library but a standalone exe. Can be fixed  
> in the future
> if there's a need.
> * doesn't generate wx/win makefiles, but... win is maintained by  
> hand anyway,
> there is no wx work at all, so the point is moot

Well, it's true that there hasn't been for the past couple months or  
so, but that's partly because wxWidgets/wxPython is being included in  
Leopard and we've been trying to polish it as much as possible before  
the final build used for Leopard is decided, and also partially  
because I wasn't sure how I wanted to proceed on the port, in terms  
of simply wrapping the CG/Cairo ports or writing a wx backend.  
However, I'm seeing some increased interest in the wx port outside of  
myself ;-), and we've recently seen the addition of a wx wrapper for  
CG/GDI+/Cairo (on Mac/Win/Linux respectively), so that's starting to  
renew interest in the idea of a wx backend. I did actually spend most  
of yesterday trying to get the sources back up to speed, and I'm  
getting close to having patches for the compilation issues I found  
ready.

So it's not a moot point for me, and in fact, although I'm working on  
Mac right now, the results of running update-file-lists.py suggest to  
me the Win port makefiles are out of sync with TOT as well. Of  
course, the reason this happens is that all source file changes must  
be made in three places (now four, actually, or five if you count  
CMake), which IMHO is getting very burdensome and inefficient. This  
is one area where wxWidgets is much nicer, as all build system  
changes only need to be made once.

I will admit that, unfortunately, it appears the Bakefile maintainer  
is swamped with other projects and deadlines, and so the project is  
not being improved at a regular pace. But outside of the  
DerivedSources issue, which was fairly easy to work around, I haven't  
had any problems and Bakefile is of course being used almost daily on  
the wxWidgets project. Furthermore, when I look at CMakeLists.txt and  
the Bakefiles, I find the Bakefiles much more readable than CMake  
scripts, and in my experience that often translates into simpler  
maintenance.

Anyway, I just wanted to clear these matters up. I know that Bakefile  
has it's issues, but I'm not totally sure the issues are really  
specific to Bakefile at all, or even if other systems provide better  
solutions out of the box. All I know is that Bakefiles + my build  
script gives me very efficient rebuilds, and wx is absolutely huge  
and certainly requires a great deal of efficiency over 3 platforms  
and multiple compilers, so I think Bakefile can certainly be used as  
an effective solution to this issue of maintaining cross-platform  
makefiles.

Regards,

Kevin

> Adding this build system to the tree doesn't mean that bakefiles have
> to go - as long as there are people maintaining those system, there's
> no harm in having more than one. I use this makefile for my builds and
> I plan on maintaining it in the future.
>
> -- kjk
>
> As a sidenote, I don't understand why most makefile generators
> (bakefile and cmake) require listing every file to build. My little
> script uses directory + list of exceptions (files *not* to compile)
> which makes for a dramatically more compact description of targets.
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at opendarwin.org
> http://www.opendarwin.org/mailman/listinfo/webkit-dev