[webkit-dev] Build system update

[Adam Barth]

On Wed, Mar 23, 2011 at 12:22 AM, Mark Rowe <mrowe at apple.com> wrote:
> On 2011-03-22, at 23:50, Adam Barth wrote:
>> On Tue, Mar 22, 2011 at 8:14 PM, Mark Rowe <mrowe at apple.com> wrote:
>>> On 2011-03-22, at 19:16, Adam Barth wrote:
>>>> WebKit-folk,
>>>>
>>>> With a bunch of help from Dimitri and Eric, we now have a functioning
>>>> GYP-based build for the Apple Mac port.  There are still a couple bugs
>>>> we need to fix before this build system is ready for production
>>>> (they're filed as blocking Bug 55018, if you're curious).  If you'd
>>>> like to try out the GYP-based build, you can invoke it (on a Mac) as
>>>> follows:
>>>>
>>>> ./Tools/Scripts/build-webkit --gyp
>>>
>>> Is this expected to work?  Running the command quoted above results in a build failure in JavaScriptCore for me (jsc is unable to find UString.h).
>>
>> I'm not sure how to test with a case-sensitive file system on Mac.  (I
>> didn't even know that was possible.)  How can I test that case?
>
> Short of formatting a volume as case-sensitive HFS+, you can use Disk Utility to create a read/write disk image containing case-sensitive HFS+ and then perform a build on that disk image.

Thanks.  This issue should be fixed as of
<http://trac.webkit.org/changeset/81759>.  Note: I wasn't able to
completely test the build on a case-sensitive file system because I
made my disk image too small.  I'll verify that the issue is fixed
tomorrow.

>>> Product names for targets are redundantly declared in the Xcode project when they're already defined in the .xcconfig files.
>>
>> My plan for this issue is to remove the product names from the
>> xcconfig files once they're not needed by the generated xcodeproj
>> files.
>
> That seems backwards.  JavaScriptCore.xcconfig defines how to create JavaScriptCore.framework.  The name of the framework (e.g., PRODUCT_NAME) is an important part of that, much like the install path, Info.plist file, etc.  I don't see how we benefit from that information being spread around.

The information about how to create JavaScriptCore.framework is
already spread around to a certain extent.  For example,
JavaScriptCore.xcconfig does not contain the list of files that need
to be built or information about the dependencies.  I'm having a hard
time seeing what the practical difference is between specifying the
product name in the xcodeproj file or the xcconfig file.

>>>> One area that I could use some help from one or more Apple folks is
>>>> making sure that this build system integrates properly with Apple's
>>>> internal build system.  Based on my (incomplete!) understanding of
>>>> Apple's internal build system, there are at least two options:
>>>>
>>>> 1) Apple's internal build system consumes Source in its entirety and
>>>> builds a master WebKit.xcodeproj (or a Makefile), which abstracts
>>>> further details about the build, such as how subsequent xcodeproj are
>>>> created or how many libraries WebKit is factored into (e.g., letting
>>>> us extract WTF from JavaScriptCore).
>>>
>>> Making that particular change is completely unrelated to GYP.
>>
>> My understanding is that it would solve the Apple internal build
>> system integration issue because the Apple internal build system would
>> transfer control to the master WebKit.xcodeproj, which could then
>> invoke GYP to generate the remaining xcodeproj files.
>
> I'm not particularly keen on a solution that involves manually invoking xcodebuild to build projects.  It becomes very complicated to ensure that the correct settings, including any overridden settings, make it down to the nested invocations of xcodebuild.

I'm not sure I've correctly communicated how I envision this approach
working.  If approach (2) doesn't work out for whatever reason, I can
build a mockup of how this would work, which will be more concrete.

>>>> 2) For each submission to Apple's internal build system, we create a
>>>> branch, generate xcodeproj files, and check them into the branch.
>>>> Apple's internal build system can then "svn export" the branch and see
>>>> xcodeproj files, as today.
>>>
>>> While this is certainly technically feasible, it would add a huge amount of overhead to the process of performing a submission.
>>
>> How often do you submit WebKit to the Apple internal build system?  If
>> that's sensitive information, I'm just looking for an order of
>> magnitude (e.g., every revision, every hour, every day, every week).
>> As a reference, the Chromium project creates one branch per day for
>> daily builds.
>
> I don't see how the frequency is relevant.

The relevancy arises from the observation that the overhead is
proportional to the frequency.  If we only submit WebKit to Apple's
internal build system once a week, then this approach won't cause too
much branch proliferation.  If, on the other hand, we're submitting
every revision, then we're talking about doubling or tripling the
revision burn rate, which might not be desirable.

>> In any case, I'm glad we've found a technically feasible solution.
>
> We've had at least one technically feasible solution from day zip: check in the generated project files.

>From my perspective, approach (2) is more desirable than checking in
generated project files because approach (2) encapsulates
Apple-internal build process to Apple folks, more specifically to the
Apple folks who interact with the Apple-internal build system.
Checking in generated project files, on the other hand, imposes a
maintenance burden on all WebKit contributors.

Adam