[webkit-dev] Moving WTF out of JavaScriptCore

Wed Feb 9 16:21:20 PST 2011

On Feb 9, 2011, at 3:11 PM, Mark Rowe wrote:

> 
> On 2011-02-09, at 14:51, Kevin Ollivier wrote:
> 
>> On Feb 9, 2011, at 2:08 PM, Mark Rowe wrote:
>> 
>>> 
>>> On 2011-02-09, at 14:05, Adam Barth wrote:
>>> 
>>>> On Wed, Feb 9, 2011 at 1:30 PM, Mark Rowe <mrowe at apple.com> wrote:
>>>>> On 2011-02-09, at 13:06, Adam Barth wrote:
>>>>>> On Thu, Dec 23, 2010 at 1:32 PM, Maciej Stachowiak <mjs at apple.com> wrote:
>>>>>>> On Dec 22, 2010, at 12:06 PM, Adam Barth wrote:
>>>>>>>> On Wed, Dec 22, 2010 at 10:40 AM, Mark Rowe <mrowe at apple.com> wrote:
>>>>>>>>> On 2010-12-22, at 10:34, Adam Barth wrote:
>>>>>>>>>> As an aside, would creating the Sources directory make it easier to
>>>>>>>>>> move WTF out of JavaScriptCore?
>>>>>>>>> 
>>>>>>>>> I don't think that the location of the source on disk is a big factor in
>>>>>>>>> WTF's presence in JavaScriptCore.
>>>>>>>> 
>>>>>>>> Oh, I thought the main involved in creating a new top-level source
>>>>>>>> directory was the main thing causing WTF to live inside
>>>>>>>> JavaScriptCore.  I know we've talked about moving it out of
>>>>>>>> JavaScriptCore for a while now.  Is that just a matter of someone
>>>>>>>> (e.g., me) doing the work or are there other limiting factors?
>>>>>>> 
>>>>>>> If we switch to a top-level Sources directory, and on the Apple-internal side switch to submitting WebKit to the OS build using this single top-level sources directory, then it's easy to make new top-level directories without disrupting anything.
>>>>>>> 
>>>>>>> If we don't do the above, we'd have to set things up so we can submit WTF separately to the build to cope, and we'd need to make it build a static library and install private headers.
>>>>>>> 
>>>>>>> So I think it is easier to make this change if we do the Sources/ change first.
>>>>>> 
>>>>>> Now that we've moved all the source code into Source, I'd like to try
>>>>>> moving WTF out of JavaScriptCore.  Is there anything on the
>>>>>> Apple-internal side that needs to happen first?
>>>>> 
>>>>> Maciej's response above mentions that we need to make some internal process changes before it's easy to add new top-level projects without disrupting things.  Unless there's some urgent reason to move WTF I'd prefer that you hold off on doing this.
>>>> 
>>>> Would it make sense to set a target date in the future for making
>>>> these internal changes?  I'm certainly not in a rush, but I'd also
>>>> like to make progress on this at some point.
>>> 
>>> I don't think that would be useful.  The changes that block this are the same changes that prevent us from removing the concept of forwarding headers. We're as interested as you are in seeing them done.
>> 
>> I'm also very interested in seeing this done, as it affects the wx port too, and I know from trying to make header changes to WTF (and watching all the breakages) that almost all ports have hacks along the lines of forwarding headers that deal with the issue of WTF being a separate project in theory, but a private implementation detail of JSC in practice. Pulling WTF out of JSC would, I think, go a long way to undoing a lot of those hacks and reducing the time and maintenance cost of making changes to WTF.
> 
> As far as I'm aware only the Mac and Windows ports have anything along the lines of forwarding headers.  No other port that I'm aware of needs anything similar.
> 
>> Is the problem here that these internal changes are very difficult and/or time consuming to implement? I wish I could help somehow because I'd much rather spend time trying to fix this problem the right way and for good than spending that time sorting out all the build breakages every time I try to do something like add a new header to WTF that I want to be widely available among ports and projects in WebKit. 
> 
> If you can point me at changes where you've had issues with forwarding headers I can provide suggestions as to how you can avoid problems with them in the future.  A quick look through SVN history doesn't reveal any changes related to forwarding headers from you.

Actually, I'm referring to what is still an open bug. (https://bugs.webkit.org/show_bug.cgi?id=27551) The issue in question is a change to try and move the definition of JSC / WTF export symbols into the headers rather than using the current export definition files. Resolving this issue is a pre-requisite for wx DRT support, and also blocks a bug related to defining export macros for C++ DOM bindings as well.

Part of what fixing this bug entailed was adding a new header to WTF, I called it wtf/ExportMacros.h, which defined WTF_EXPORT and WTF_IMPORT so we could consolidate the export symbol attribute definition macros into one place in the tree, instead of copying and pasting the same logic (just with different XYZ_EXPORT and IMPORT names) into every config.h file. However, when I tried adding the WTF header, almost every port failed with some error, and it often had to do with header availability issues. (e.g. WebCore has access to this WTF header, but WebKit, or DRT, or JSC.exe doesn't, and won't unless you add it to some copied / forwarded header list). 

FWIW, I did get around to fixing most of the ports, but the reason this never got landed is that I ran into a roadblock with the Win port. While the Windows build worked on my machine, the EWS Windows build failed, and at the time I worked on this patch the Windows EWS build did not actually give you the output as to what exactly failed. So all I knew was that something was failing, and I couldn't reproduce the failure on my local machine doing a stock build-webkit build. So I was kind of stuck as to what to do to make everything green. The lack of Windows EWS log reporting has since been resolved, but I hadn't had time to look back into dealing with the symbol issue again until now. 

I'm not saying that this can't be fixed without moving WTF, and I'd definitely appreciate any and all ideas of how to better approach solving this problem, but it seems to me that if WTF were defined quite clearly as a separate library, one which each port would make available to all projects in their build and which did not have the sub-project independence policy that projects like JSC have, doing something like adding a header would have been much more straightforward and most likely would not have resulted in numerous build issues arising because each port (and sometimes each project within the port) needed a separate fix for making that header available.

Kevin

> - Mark
> 