[webkit-dev] Common build system (was Re: WebKit Wishes)

[Dirk Pranke]

On Wed, Jan 30, 2013 at 4:15 PM, Maciej Stachowiak <mjs at apple.com> wrote:
>
> On Jan 30, 2013, at 3:24 PM, Dirk Pranke <dpranke at chromium.org> wrote:
>
>> On Wed, Jan 30, 2013 at 1:50 PM, Filip Pizlo <fpizlo at apple.com> wrote:
>>> Thanks for sharing this.
>>>
>>> On Jan 30, 2013, at 1:28 PM, Eric Seidel <eric at webkit.org> wrote:
>>>
>>> I wish we only had one build system (it were easy to add/remove/move files).
>>>
>>> I believe changes like http://trac.webkit.org/changeset/74849 are an
>>> unhealthy sign for the project.  Adam is not the only person who has chosen
>>> to empty files instead of removing them.  The pain of updating 8 build
>>> system is so great, we jump through hoops to avoid it.  Which means it took
>>> us months to move JavaScriptCore/wtf to WTF, and will take us years to kill
>>> WebCore/platform.
>>>
>>>
>>> +1
>>>
>>> This is a hard problem.  It is a problem worth solving.
>>>
>>> Do you have more thoughts on this, particularly since you know quite well
>>> how both Xcode and gyp work?
>>>
>>> I suspect this is one of those things that it would be hard to achieve
>>> consensus on since there are so many stakeholders.  But it may be fruitful
>>> to have a "what if" discussion about what this might look like.
>>>
>>
>> I think we can solve this problem if we agree that we want to. I think
>> we haven't in the past mostly because we couldn't reach a consensus
>> that it was worth solving enough to really try.
>>
>> I would love to see this fixed and would be glad to work on it. I
>> think we should at least pursue this far enough to fully understand
>> what our options are and what the costs and tradeoffs might be; does
>> anyone disagree, and is anyone else willing to help pitch in?
>>
>> I think there are several possible ways we could solve this. One would
>> be to switch to a common meta-build system. My understanding is that
>> Apple's internal production build processes impose certain constraints
>> here that I don't fully understand, but I know we've discussed the
>> possibility of checking in generated project files as a workaround.
>> Maybe there are other options as well to those constraints? I would
>> love to discuss this further w/ someone from Apple ...
>
> It's far simplest for us if:
> (a) There is an Xcode project (or a Makefile) that builds the Mac port checked in to source control.
> (b) The generated project invokes only tools that are part of the default Mac OS X install.
>
> It may not be completely impossible to violate these requirements but it will require a lot of bureaucracy.
>
>> (Also, just to get this out of the way, I don't think gyp needs to be
>> the solution).
>>
>> Another alternative would be to write a script that did support at
>> least the common use cases (add/move/delete files). There have been
>> attempts in the past, but they have foundered on at least some
>> perceived skepticism over how well this would work w/ XCode. That
>> said, I don't think we've really pushed this to see. At some point
>> this script might turn into a meta-meta-build system, which might be
>> silly but also be the shortest path to the finish line.
>>
>> I suggest if there is interest in this we can start a new thread to
>> discuss further ...
>
> My preference would be to use a common meta-build system with a comfortably human-readable and human-editable syntax, and checked in generated project files for those ports that need them.
>
> I think a key to making this work is to get Chromium and the Apple Mac port onto a common build system, which will probably require both Google and Apple ponying up at least one person to work on this project for a reasonable period of time.
>
> I think the plausible meta-build-systems to use would be CMake and Gyp, but in both cases it may be necessary to modify them to work well for everyone.
>

Premake might also be an option, though I wouldn't necessarily vote
for it. Gyp's syntax is ... awkward ... but apart from that might just
work for checking in generated apple mac xcode projects. We should try
it (shouldn't be too hard, since abarth had it working at one point).
I would consider changing or improving gyp's syntax to be on the table
if it was needed to reach the goal.

CMake was originally considered a non-starter for chromium since the
XCode and VS projects it produced didn't look or feel like native
projects. However, we've since switched to ninja for most things so
I'm less sure how important this is now. I've never been a huge fan of
the CMake syntax but I haven't used it enough to really have an
informed opinion.

Regarding "(b) The generated project invokes only tools that are part
of the default Mac OS X install": invoking tools that are checked into
the WK repo is also possible, right, since we invoke scripts now? So,
hypothetically, could we check in a copy of the ninja binary and build
with that?

(Of course, there might be other reasons why ninja might not work, and
I'm not suggesting you'd want to do that, just trying to understand
the constraints).

> I'd also like to add that I think a key related issue to common build system is common feature configuration. The many different ways ports control their feature flags is super confusing. I've long wanted to implement common configuration management, but have not had time.
>

I think this would be a nice related project, but I wouldn't
necessarily want to lump the two goals together without understanding
what we'd be shooting for first. From what I know, CMake's support for
feature configuration is much more mature than what you can do w/ GYP.
It could certainly be a good motivator, though.

-- Dirk