OK to flatten WTF's header directory?
Hi webkit-dev, I’m working on fixing some ambiguities in our Xcode projects to permit adoption of Xcode’s new build system and better parallelize our builds. I noticed that WTF’s headers (/usr/lib/include/wtf) are atypical in that they aren’t copied into a single directory – they’re deeply nested and mirror the same directory hierarchy as exists in WTF’s sources. Is this intentional, and if not, would there be opposition to me flattening them into a single directory? Flattening them makes it easier to explain to Xcode what headers go where, which is useful for tracking dependencies between targets and ensuring proper build order. Headers would still be in their same source locations (alongside implementation files) but would be copied to the top-level /usr/lib/include/wtf directory at build time. The only other place in our projects [1] I’m aware of with deeply-nested headers is PAL, and there’s a bug to change that: https://bugs.webkit.org/show_bug.cgi?id=175705 <https://bugs.webkit.org/show_bug.cgi?id=175705> -Elliott [1]: libwebrtc also produces deeply-nested headers, but since it’s a third-party project, I don’t think our header conventions should apply.
On Tue, Feb 1, 2022 at 12:30 PM Elliott Williams via webkit-dev < webkit-dev@lists.webkit.org> wrote:
I’m working on fixing some ambiguities in our Xcode projects to permit adoption of Xcode’s new build system and better parallelize our builds. I noticed that WTF’s headers (/usr/lib/include/wtf) are atypical in that they aren’t copied into a single directory – they’re deeply nested and mirror the same directory hierarchy as exists in WTF’s sources.
What do you mean by this? Could you give us examples? Is this intentional, and if not, would there be opposition to me flattening
them into a single directory? Flattening them makes it easier to explain to Xcode what headers go where, which is useful for tracking dependencies between targets and ensuring proper build order. Headers would still be in their same source locations (alongside implementation files) but would be copied to the top-level /usr/lib/include/wtf directory at build time.
The only other place in our projects [1] I’m aware of with deeply-nested headers is PAL, and there’s a bug to change that: https://bugs.webkit.org/show_bug.cgi?id=175705
-Elliott
[1]: libwebrtc also produces deeply-nested headers, but since it’s a third-party project, I don’t think our header conventions should apply. _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
On Feb 1, 2022, at 12:30 PM, Elliott Williams via webkit-dev <webkit-dev@lists.webkit.org> wrote:
Hi webkit-dev,
I’m working on fixing some ambiguities in our Xcode projects to permit adoption of Xcode’s new build system and better parallelize our builds. I noticed that WTF’s headers (/usr/lib/include/wtf) are atypical in that they aren’t copied into a single directory – they’re deeply nested and mirror the same directory hierarchy as exists in WTF’s sources.
Is this intentional, and if not, would there be opposition to me flattening them into a single directory? Flattening them makes it easier to explain to Xcode what headers go where, which is useful for tracking dependencies between targets and ensuring proper build order. Headers would still be in their same source locations (alongside implementation files) but would be copied to the top-level /usr/lib/include/wtf directory at build time.
Question: If they're flattened in the SDK, and not flattened in the source tree, which include path do we use when including a WTF header in e.g. WebCore? Right now you say `#include <wtf/text/WTFString.h>`. In your world, would you say `#include <wtf/WTFString.h>`? If no, how does that work in the case where you only have WTF from the SDK? If yes, how does that work for the other build systems that are building with the full source tree? (maybe we have all directories in WTF listed as header search paths, or something? or maybe they look in `usr/lib/include/wtf` in the build directory?)
The only other place in our projects [1] I’m aware of with deeply-nested headers is PAL, and there’s a bug to change that: https://bugs.webkit.org/show_bug.cgi?id=175705 <https://bugs.webkit.org/show_bug.cgi?id=175705>
-Elliott
[1]: libwebrtc also produces deeply-nested headers, but since it’s a third-party project, I don’t think our header conventions should apply. _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
On Feb 1, 2022, at 22:12, Tim Horton <timothy_horton@apple.com> wrote:
Question: If they're flattened in the SDK, and not flattened in the source tree, which include path do we use when including a WTF header in e.g. WebCore?
Right now you say `#include <wtf/text/WTFString.h>`. In your world, would you say `#include <wtf/WTFString.h>`?
Yes, exactly.
If yes, how does that work for the other build systems that are building with the full source tree? (maybe we have all directories in WTF listed as header search paths, or something? or maybe they look in `usr/lib/include/wtf` in the build directory?)
My understanding is that "maybe we have all directories in WTF listed as header search paths” is how we’ve set up WebCore, JSC, and other frameworks which already flatten their headers in their build directory. For example, JSC adds all its header directories here <https://trac.webkit.org/browser/webkit/trunk/Source/JavaScriptCore/CMakeList... <https://trac.webkit.org/browser/webkit/trunk/Source/JavaScriptCore/CMakeLists.txt#L16>>. IIUC, these paths become part of the JSC’s target that is used downstream.
Long ago, I originally created the forwarding headers to bridge the gap between framework-style includes that those of us at Apple wanted to do, where headers are flattened and you write #include <FrameworkName/HeaderName.h>, and Unix-style installed libraries, where things are not flattened. I wanted us to be able to write include statements in WebKit in the traditional non-Apple-framework style so they would work on those other platforms, and the forwarding headers were originally just for the Apple platforms, making those includes work with the framework structure provided automatically by the Apple framework build system. Those original forwarding headers were not copies of the headers, they were simply files “in the right place” with include statements in them. I’m not sure at what point along the way we started making copies of headers, and we may be solving other problems with that. This proposal, to flatten the WTF headers, seems to be a step in of the opposite direction, making the non-Apple platforms do something akin to framework-style including. Not sure I understand the pros and cons are of that. I am currently suffering when working on WebKit development because of the copies of headers made by the build system. I can’t count the number of times I’ve edited a WTF header and then later realized I had edited the copy, not the original. I would very much like a solution that resolved that problem. I use Xcode features, and it seems the copies are what Xcode thinks are the “real” headers. — Dar
Thanks for the detailed response! I should be clear that I don’t have a strong opinion for or against flattened headers, I’m just interested in ironing out a perceived inconsistency.
On Feb 3, 2022, at 15:31, Darin Adler <darin@apple.com> wrote:
Long ago, I originally created the forwarding headers to bridge the gap between framework-style includes that those of us at Apple wanted to do, where headers are flattened and you write #include <FrameworkName/HeaderName.h>, and Unix-style installed libraries, where things are not flattened. I wanted us to be able to write include statements in WebKit in the traditional non-Apple-framework style so they would work on those other platforms, and the forwarding headers were originally just for the Apple platforms, making those includes work with the framework structure provided automatically by the Apple framework build system.
IIUC, what we have today is the opposite situation. JavaScriptCore, WebCore, et al. have flattened headers, and it’s the _other platforms_ whose build systems do extra work to accommodate flattened includes. WTF and PAL are the exceptions.
This proposal, to flatten the WTF headers, seems to be a step in of the opposite direction, making the non-Apple platforms do something akin to framework-style including. Not sure I understand the pros and cons are of that.
The factors motivating me are… - Consistency with other targets: It’s not clear to me why JSC and WebCore have flattened their headers but WTF hasn’t. - Pleasing Xcode’s build system, which _really_ wants a target’s headers to always be copied to the same directory. Right now, WTF hacks around this by using an rsync script to copy its headers because Xcode’s native copy files phase doesn’t provide enough flexibility. Build system limitations shouldn’t influence ergonomic decisions, so perhaps you should disregard the latter.
I am currently suffering when working on WebKit development because of the copies of headers made by the build system. I can’t count the number of times I’ve edited a WTF header and then later realized I had edited the copy, not the original. I would very much like a solution that resolved that problem. I use Xcode features, and it seems the copies are what Xcode thinks are the “real” headers.
If what you’ve experienced is unique to WTF, I’d be suspicious of the non-standard approach we’ve taken to copying WTF’s headers I mentioned above.
On Fri, Feb 4, 2022 at 8:31 AM Darin Adler via webkit-dev < webkit-dev@lists.webkit.org> wrote:
Those original forwarding headers were not copies of the headers, they were simply files “in the right place” with include statements in them. I’m not sure at what point along the way we started making copies of headers, and we may be solving other problems with that.
The Apple Internal Build System separately compiles out AppleWin port's each module, so the forwarding headers don't work, they need to copy headers. https://bugs.webkit.org/show_bug.cgi?id=180064 <https://bugs.webkit.org/show_bug.cgi?id=180064> https://bugs.webkit.org/show_bug.cgi?id=180063
participants (5)
-
Darin Adler
-
Elliott Williams
-
Fujii Hironori
-
Ryosuke Niwa
-
Tim Horton