[webkit-dev] [webkit meeting notes] build systems

[Kevin Ollivier]

Hi Marc-Antoine,

On Apr 17, 2010, at 11:26 AM, Marc-Antoine Ruel wrote:

> 2010/4/17 Kevin Ollivier <kevino at theolliviers.com>
> Hi Maciej,
> 
> On Apr 16, 2010, at 3:34 PM, Maciej Stachowiak wrote:
> 
> >
> > On Apr 16, 2010, at 8:09 AM, Nikolas Zimmermann wrote:
> >
> >>
> >> Am 16.04.2010 um 16:44 schrieb Adam Treat:
> >>
> >>> I am very skeptical that it is feasible to write a gyp generator that would
> >>> output QMake files.  There is a log of magic in those QMake files.  My sense is
> >>> that it would not be trivial by any means.
> >>>
> >>> Plus, I don't like the idea of a meta-meta generators.  Seems way to mickey-
> >>> mouse to me.
> >>
> >> Agreed to a certain degree. Using gyp/whatever to generate qmake files, to generate Makefile/Xcode files etc seems akward to me as well.
> >>
> >> What we really need to resolve is adding/removing files from compilation, that's the most common
> >> task that has to be done in 5+ build systems at the moment.
> >
> > Besides adding, removing and renaming, the other thing that's really hard is adding a new generated source rule. Although this is not needed as frequently, I think anyone adding a new code generator script that has to work across all WebKit ports would have a hellish time of it right now.
> >
> > If I had to do it myself, I would just skip any ports that don't use DerivedSources.make.
> >
> >
> >> So I have a new proposal:
> >>
> >> 1) Maintain a list of headers/source files to be compiled for ALL platforms (ie. dom/Node.cpp, etc..)
> >>
> >> 2) Keep all existing Makefile.am, WebCore.pro etc files as "templates", ie. WebCore.pro.template, with a special
> >>   variable somewhere marking the $$HEADER_LIST$$ and the $$SOURCE_LIST$$
> >>
> >> 3) Use a script that generates individual build files (eg. WebCore.pro) from WebCore.pro.template, it only
> >>   needs to insert the file list with the correct syntax in the correct places
> >>
> >> 4) Keep all platform specific files to be compiled in the individual build system files (eg. WebCore.pro.template)
> >>
> >> I think we'll never find a consensus on a single build system, there are too many different needs. I only care
> >> about the most repetitive work in order to keep the build system up2date: adding/removing cross-platform files.
> >
> > I think the proposal above does not handle the derived sources problem. It also doesn't handle files that are shared between multiple ports but not all ports. It also doesn't provide project files that are directly usable by IDEs, on platforms where that is the standard way to do development.
> 
> Converting, say, a JSON list of files to some XML-based output format would not be difficult at all (and I
> 
> Like this?
> http://trac.webkit.org/browser/trunk/WebCore/WebCore.gypi
> 
> - It is currently not JSON compliant. It's in fact a python file but that can be fixed: s/'/"/g and removing the extra commas should be sufficient.
> - It is currently chromium specific, that's what I meant by "de-chromification" of the gyp files. It's mainly adding more stuff and fixing the regexp if I'm not mistaken. I don't mind if it doesn't become the golden file, the goal is just to hopefully reduce the number of build system, nothing more.

Yes, precisely why I mentioned JSON and later gyp, though I don't know if Chromium-specific is the right word here. It even has wx port files in it, which I don't think are built by Chromium. :) I suppose you somehow filter out which ones Chromium should build after the fact? If so, where does that logic reside?

Anyway, the fact that this file is actually Python (I had forgotten the format was JSON-like rather than straight JSON) makes things even better, as we only really need to handle export now, and not parse some import file list. i.e. for WebKitTools/Scripts/update-sources-list.py, getWebCoreFilesDict() basically becomes a very small script that execfile's the gypi file, then we run whatever filtering mechanism on it (waf has a list of ports we could use to do the filtering that I could probably move into WebKitTools/Scripts, if we don't have a pre-made Chromium solution here), and then passes the final source list along to generateWebCoreSourcesGTKandQT to generate the sources / includes for those platforms, and actually this solution should work for Android.mk and possibly jam too, as their syntax looks largely similar. Then we'd add some XML parsing code to grab the node for common file groups and update them for the MSVC projects, and then I think that mostly leaves XCode, which I think would be pretty similar in nature. 

As long as people are willing to test out this solution with their build systems and help with any debugging, I would be willing to help out in hacking it together, though I can't promise anything in the way of time since this is not an immediate concern for me personally.

Thanks,

Kevin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20100417/b9f2eca6/attachment.html>