[webkit-dev] Common build system (was Re: WebKit Wishes)

[Eric Seidel]

+1

Ninja is beyond-words amazing.  http://martine.github.com/ninja/  For
better or worse, it is not designed to use human-editable build files,
but rather to be used by a meta build system, like GYP or CMake.  So
using ninja is really an orthogonal discussion to the "single build
system" discussion for WebKit. :)

Were the WebKit project to convert to using a single meta-build
system, ninja would become an option many users might choose.  I'm
told most Chromium hackers have GYP set to output ninja files these
days, with the exception of some folks who still want the MSVC build
environment. For WebKit ports already using CMake, they should
definitely try ninja today!


Anyway, my wish was not about arguing for a specific build solution.
I'm instead noting that for the project to continue to move quickly,
we need to stop needing to edit 8 build systems for every file
move/addition.  Whether that's GYP or CMake or something else, I don't
really care.  Adam and I tried GYP-for-WebKIt a while back.  But any
of these solutions will require buy-in from Apple, as they will have
to do the largest amount of work converting to use something other
than XCode.


On Sat, Feb 2, 2013 at 8:20 PM, Nico Weber <thakis at chromium.org> wrote:
> On Sat, Feb 2, 2013 at 4:58 PM, Adam Barth <abarth at webkit.org> wrote:
>> Ninja has extremely fast incremental builds and can be generated by
>> GYP.  Here are some stats from a year ago:
>>
>> https://plus.google.com/101038813433650812235/posts/irc26fhRtPC
>>
>> Ninja has gotten even faster since then.  If you're interested in
>> trying it out, you can play around with incremental builds of the
>> Chromium port on Mac or Linux.
>
> You can also look at the build output from the chromium bots.
>
> Empty build in 1s:
> http://build.webkit.org/builders/Chromium%20Linux%20Release/builds/66807/steps/compile-webkit/logs/stdio
> Build with a few files changed in 15s:
> http://build.webkit.org/builders/Chromium%20Linux%20Release/builds/66800/steps/compile-webkit/logs/stdio
>
> …and this is on fairly slow bots. On my SSD-equipped laptop, I can do
> incremental rebuilds of all of chrome after touching one (cpp or mm)
> file in 2-6s.
>
>>
>> Adam
>>
>>
>> On Fri, Feb 1, 2013 at 4:58 PM, Balazs Kelemen <kbalazs at webkit.org> wrote:
>>> I think one important aspect of build systems was not considered yet int
>>> this conversation: speed. The time an incremental build takes has a great
>>> effect on developer productivity. I don't think any of the meta-build
>>> systems we use does a great job here - although I only have experiences with
>>> qmake, cmake and autotools (and I don't have an SSD that could help
>>> somewhat).
>>>
>>> The technic I found useful here is to avoid calling build-webkit always and
>>> instead just rebuild the subproject you have edited, so I think it is
>>> important to have a build system that supports it. Let me share my
>>> experiences here.
>>>
>>> With qmake nowadays this work perfectly. The developer build is producing a
>>> shared library for every subdir (WTF, JavaScriptCore, WebCore, WebKit2),
>>> which means you only need to call make in the specific subdirectory (i.e. if
>>> I only touched WebKit2 files I do "make -C
>>> WebKitBuild/Release/Source/WebKit2" which is pretty quick). Still WebCore is
>>> so big that make is quite slowly find out the files you actually edited and
>>> need to be rebuilt. What could help here is to devide WebCore into smaller
>>> parts, like the ongoing work of extracting Platform. Maybe the next logical
>>> candidate could be svg (I don't have real knowledge about how feasible it
>>> is).
>>>
>>> Note that I don't come up with qmake because I would like to recommend it as
>>> the one and only build system (in fact it has a ridiculously inconvenient
>>> syntax, and a lot of bugs), just as an example.
>>>
>>> With Cmake fast incremental rebuilds are also possible, maybe in a bit more
>>> complicated way. When working with the EFL port I found a quick rule for
>>> WebKit2 in the generated makefile. Although the makefiles are usually call
>>> back to Cmake, and make is not faster than build-webkit, if you use the
>>> special fast target, which is something like eflWebKit/fast (i.e. make -C
>>> WebKitBuild/Release/Source/WebKit2 eflWebKit/fast), it will not check
>>> dependencies but just rebuild the files that have changed. I did not find a
>>> similar thing for WebCore, I guess because it is not built as a shared
>>> library.
>>>
>>> What I dislike in Cmake is that I am disappointed by how slow a normal
>>> incremental build with it (i.e. build-webkit). qmake is not faster at all,
>>> but it generate plain makefiles that typically no call back to qmake if not
>>> specified explicitly to do so, and directly calling make is faster, yet it
>>> can handle most of the non-trivial changes, for example editing a generator
>>> file. I don't know gyp, so I wonder about how would it do in this comparison
>>> (but I guess it generates simple makefiles as well, so it's similar than
>>> qmake in this manner.)
>>>
>>> I hope I added something to this conversation that is worth to consider with
>>> my late nightly brain dump.
>>>
>>> -kbalazs
>>>
>>>
>>> _______________________________________________
>>> webkit-dev mailing list
>>> webkit-dev at lists.webkit.org
>>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>>
>> On Fri, Feb 1, 2013 at 4:58 PM, Balazs Kelemen <kbalazs at webkit.org> wrote:
>>> I think one important aspect of build systems was not considered yet int
>>> this conversation: speed. The time an incremental build takes has a great
>>> effect on developer productivity. I don't think any of the meta-build
>>> systems we use does a great job here - although I only have experiences with
>>> qmake, cmake and autotools (and I don't have an SSD that could help
>>> somewhat).
>>>
>>> The technic I found useful here is to avoid calling build-webkit always and
>>> instead just rebuild the subproject you have edited, so I think it is
>>> important to have a build system that supports it. Let me share my
>>> experiences here.
>>>
>>> With qmake nowadays this work perfectly. The developer build is producing a
>>> shared library for every subdir (WTF, JavaScriptCore, WebCore, WebKit2),
>>> which means you only need to call make in the specific subdirectory (i.e. if
>>> I only touched WebKit2 files I do "make -C
>>> WebKitBuild/Release/Source/WebKit2" which is pretty quick). Still WebCore is
>>> so big that make is quite slowly find out the files you actually edited and
>>> need to be rebuilt. What could help here is to devide WebCore into smaller
>>> parts, like the ongoing work of extracting Platform. Maybe the next logical
>>> candidate could be svg (I don't have real knowledge about how feasible it
>>> is).
>>>
>>> Note that I don't come up with qmake because I would like to recommend it as
>>> the one and only build system (in fact it has a ridiculously inconvenient
>>> syntax, and a lot of bugs), just as an example.
>>>
>>> With Cmake fast incremental rebuilds are also possible, maybe in a bit more
>>> complicated way. When working with the EFL port I found a quick rule for
>>> WebKit2 in the generated makefile. Although the makefiles are usually call
>>> back to Cmake, and make is not faster than build-webkit, if you use the
>>> special fast target, which is something like eflWebKit/fast (i.e. make -C
>>> WebKitBuild/Release/Source/WebKit2 eflWebKit/fast), it will not check
>>> dependencies but just rebuild the files that have changed. I did not find a
>>> similar thing for WebCore, I guess because it is not built as a shared
>>> library.
>>>
>>> What I dislike in Cmake is that I am disappointed by how slow a normal
>>> incremental build with it (i.e. build-webkit). qmake is not faster at all,
>>> but it generate plain makefiles that typically no call back to qmake if not
>>> specified explicitly to do so, and directly calling make is faster, yet it
>>> can handle most of the non-trivial changes, for example editing a generator
>>> file. I don't know gyp, so I wonder about how would it do in this comparison
>>> (but I guess it generates simple makefiles as well, so it's similar than
>>> qmake in this manner.)
>>>
>>> I hope I added something to this conversation that is worth to consider with
>>> my late nightly brain dump.
>>>
>>> -kbalazs
>>>
>>>
>>> _______________________________________________
>>> webkit-dev mailing list
>>> webkit-dev at lists.webkit.org
>>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev