[webkit-dev] Reducing / removing use of Makefile based DerivedSources.make

[Maciej Stachowiak]

> On Oct 17, 2020, at 10:00 AM, Sam Weinig <weinig at webkit.org> wrote:
> 
> Hi webkit-dev,
> 
> I’d like to propose, and gauge feedback on, reducing (with the goal of ultimately removing) the use of Makefile based DerivedSources.make.
> 
> My understanding is that currently only the Xcode based ports still use DerivedSources.make, as all the CMake based ones have moved derived source generation to within CMake, so that should limit the scope of who this might affect.
> 
> 
> Why do we use Makefiles today?
> 
> While I can’t recall the initial reasons for using Makefiles for derived sources, over the years there have been a number of advantages to it that I do know of. One clear advantage, that is no longer applicable, was code sharing, as earlier in the project, at least the Apple Windows port did utilize these Makefiles. Another was to work around some limitations in what dependencies Xcode was able to track with build rules. It seems at least some of the problems with build rules are no longer an issue, as we can now specify inputs to build rules, but I don’t if other problems will still be there, but for some prototyping I did, nothing yet has come up.

I think I might be responsible for this, and I think the reason was that at the time, Xcode could not properly handle derived source dependencies, and ended up either gratuitously rebuilding, or miscomputing by failing to regenerate a source, or failing to rebuild it when needed. It’s possible, maybe even likely Xcode handling of derived sources has improved since then. But the last attempt to do this via Xcode caused subtle broken build issues and/ir gratuitous rebuilding, so we’d have to validate that it doesn’t have these problems any more.

DerivedSources.make is also more human-editable without having to do so via the Xcode GUI.


> 
> 
> What would we move to?
> 
> As this only affects the Xcode based ports, we would move to distinct script phases and build rules in the Xcode project.  
> 
> 
> Why make this change? What’s the benefit?
> 
> There are few reasons to consider this. One advantage is simplifying the build system. Rather than two dependency systems (one for Xcode, one for the Makefile) we reduce it down to one. And with additional knowledge of the stages and dependencies, Xcode could potentially parallelize more phases. We would would also save some time by avoiding invoking make in the first place. 
> 
> We also have a bit of an issue with make itself, as due to system requirements, we are forever stuck with Make 3.81, which is coming up on being 15 years old. More than once in the last year I have tried to troubleshoot makefile issues, looking for resources on the web, only to be stymied because the solutions I found required newer make.

One question in my mind is whether this would potentially lead to faster builds. If so, and the reliability problems from older Xcode’s are gone, then this would probably be a worthwhile change. But see below...

> 
> 
> What are the downsides?
> 
> One potential downside will be that it will be a bit harder for those without Xcode to add new types of derived sources. I am not sure how much a real problem in practice this will be, as editing project.pbxproj files is already required for just adding new files, but I want to call it out anyway.
> 
> 
> What are your thoughts on this? Are there additional reasons we might want to stick with or move away from Makefile based derived sources?

One additional though, I think we hope to move the Apple-maintained ports to CMake, so the value of changes to the Xcode-based build system may be limited. We think this could speed up both incremental and full builds significantly.

 - Maciej


> 
> Thanks,
> -Sam
> 
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20201112/3954bdf7/attachment.htm>