[webkit-dev] WebKit Wishes

[Eric Seidel]

On Wed, Jan 30, 2013 at 1:50 PM, Filip Pizlo <fpizlo at apple.com> wrote:
> Thanks for sharing this.
>
> On Jan 30, 2013, at 1:28 PM, Eric Seidel <eric at webkit.org> wrote:
>
> I wish we only had one build system (it were easy to add/remove/move files).
>
> I believe changes like http://trac.webkit.org/changeset/74849 are an
> unhealthy sign for the project.  Adam is not the only person who has chosen
> to empty files instead of removing them.  The pain of updating 8 build
> system is so great, we jump through hoops to avoid it.  Which means it took
> us months to move JavaScriptCore/wtf to WTF, and will take us years to kill
> WebCore/platform.
>
>
> +1
>
> This is a hard problem.  It is a problem worth solving.
>
> Do you have more thoughts on this, particularly since you know quite well
> how both Xcode and gyp work?
>
> I suspect this is one of those things that it would be hard to achieve
> consensus on since there are so many stakeholders.  But it may be fruitful
> to have a "what if" discussion about what this might look like.

This has broken off into a separate thread.  If I were king of the
project, I would declare GYP as the one-true solution, not because I
like GYP, but because it has solved the problem for 50% of WebKit
developers, and seems reasonable for solving it for the other 50%.

I'm not king :) and thus would not wish to decree any specific
solution.  I think what really matters here is what multi-port
solution Apple believes they could use.  It's possible for Chromium to
move away from GYP.  I don't personally care too much what meta-build
system we end up using, so long as we end up with one.

I do not think that a script to add/move/remove files/folders is a
very robust solution, unfortunately.  Leaving a meta-build system as
the only obvious option in my mind.


> I wish I felt like reviewers understood/trusted each other more.
>
> I’ve worked at both Apple and Google.  The WebKit community is full of
> brilliant engineers.  Yet I frequently feel a lack of trust in my (or
> others) judgement, or witness hot-headed remarks on bugs, lists or IRC.  I
> don’t think it’s that people don’t trust me after nearly 8 years (!?) on
> this project, but rather that we forget, or fail to communicate trust among
> ourselves.  Social problems are perhaps harder to solve for us technical
> types, but I worry that for many of us it’s just become “us” and “them” and
> we’ve stopped trying.
>
>
>
> I wish it were easy to work on feature branches.
>
> We have no good solution for features.  For one-patch features, you do them
> on your own.  For larger, you maybe use github or most likely you just land
> on trunk behind a #define.  None of these have worked well.  Some of this is
> the limits of SVN, but it should be trivial for someone to work on a new
> feature a long time, w/o endangering trunk or having massive merge pain
> every day.  Other projects can do this.  So should we.  This is both
> impeding progress, and destabilizing trunk.
>
>
> I've done this for JSC JIT optimization work before, and I think it worked
> quite well.  The pain of merging the branch back into trunk was bad, but the
> reward was that I had ~2 months of time where I didn't have to worry about
> breaking other people's junk.  Merging only took a week.  One week of pain
> in return for 2 months of bliss?  I'll take that.

Certainly.  I much prefer to work on branches.  Our current lack of
official branching solution (and tools) has left everyone to go their
own.  When I use GitHub for branching it feels like a very private
fork.  It's difficult for the rest of the project to know about or
follow my work.  Meaning that it's easy for me to go off into the
crazy-weeds the longer I'm out on my own branch. :)

Some of this is risks intrinsic with branching.  But imagine a world
in which every bug fix was its own branch?  Imagine a world where
branches were merged into trunk instead of committed as patches?  Some
of this is crazy-talk, I understand. :)  But I think we can do better
than our current tooling/process. :)

> I concur that we can do better on this.  I would especially love to see
> feature branching and merging follow a review process as if it were on
> trunk.  Creating a branch involves a bug and a review.  Patches that land on
> the branch are reviewed.  Merging is either reviewed or rubber stamped.

Correct!  Having branching always be off somewhere else is OK, but
lacks the connection to the rest of the project (and the review
process).

> I wish we didn’t have to worry about platforms we couldn’t test.
>
> It can’t be the job of the core maintainers to care about all the peripheral
> ports which contribute very little core code. Our code needs to be
> structured in such a manner that its easy for the core to march forward,
> while letting the ports catch up as they need to asynchronously.  Platform
> support code shouldn’t even need to be in webkit.org!  Porting webkit.org’s
> platform abstractions should be trivial, but core developers (which probably
> 90% of them use only 2 ports Mac WK2 + Chromium Linux) shouldn’t need to
> worry about keeping all ports working.
>
>
> +1
>
>
>
> I wish that the tree always built and tested cleanly.
>
> Other (much larger) projects than WebKit accomplish this.  Yet somehow
> Google pays 2 full-time engineers to watch our bots and yet we fail.  I know
> other companies do similar.  Automated rollouts is one solution.
> Branched-based development, or trybots are others.  But at the size and
> scale we’re at now, every minute of a broken tree, is 100x or more minutes
> of potentially lost developer productivity.
>
>
> I actually think this just goes along with the territory.  Browser engines
> are hard.  Browser engines that achieve the configurability and portability
> of WebKit: even harder.
>
> Maybe I'm naive but I think we're doing quite well on this front given the
> circumstances.  I also suspect that this problem ought to be diminished with
> your other suggestions, particularly having more branch-based development,
> and especially if there was a magical way to simplify building.

I think some of this is all tied in my head with my evolving belief
that WebKit allows porting at the wrong layer.

I think it is very hard to keep every WebKit port green when each one
ties in differently to our loader architecture. :)

I think that some of our pain traces back to architectural mistakes
from years ago, which have ports tying in at the WebCore platform
layer, instead of having WebCore only have one true way to talk to the
rest of the world (as chromium's Platform layer, and WK2
PlatformStrategies are closer to).

> I wish I felt like I could follow what was going on (and trust WebKit to
> guard the web, instead of depending on Apple or Google).
>
> We’re the leading browser engine, with hundreds of committers, any of whom
> can add an API to 50% of internet browsers with a single commit.  I wish we
> had a public process for feature/web-api review.  I wish I felt like both
> major companies were willing participants in such.  (Google has an internal
> process, but it sees limited use, in part because it’s powerless -- a ‘yes’
> from our process is not a ‘yes’ from WebKit.)  I want to feel like I can
> better observe and participate in the development of our web-api (and trust
> that it’s being done well!) without scanning every changeset just to be able
> to comment post-facto.  (This is also reflected in the fact that the
> features enabled by the major Apple or Google ports are wildly different,
> with seemingly little rhyme or reason.)
>
>
> This is made harder by the fact that a lot of the "API" is poorly specified
> to begin with, and the things we have to do to mold the API into a form
> where it doesn't break the web leads to dark and difficult to grok corners.
> I think a lot of this goes with the territory and perhaps just being more
> cognizant of the implications of the things we do is the best solution.

I think we also just don't try hard enough.  Although we all live in
the same repository, all the ports ship very divergent sets of
features.  Maybe this is good?  But personally I think it's bad for
the Web.

Even just having a single way for all features to be enabled/disabled
would give a single file for interested parties to watch in the
repository. :)

Moving all of the idl files into a separate directory, would give a
single directory for interested parties to audit for changes to the
web platform.

I think we really need to step up our game here, to prevent "hurting"
the web with both internal and external feature fragmentation as our
project and collective market shares get larger.

> I wish WebCore was not trapped by Mac WebKit1’s legacy designs.
>
> WebKit2 is obviously a step towards the future.  But until we can kill the
> Widget tree, the insanely fragile loader complexity, and the limits imposed
> by the least-common-denominator on classes like ResourceRequest, we’re still
> trapped in the past. One of the things I’ve learned in working on Chromium,
> is that we were wrong many years ago to fold our platform abstraction
> (Qt-implementation) and khtml into one library.  In a sand-boxed
> multi-process world, the rendering library is just a hunk of code running
> the same on every platform.  And platform code has no place in our core
> engine code (aka WebCore).
>
>
> I write less out of pain, and more out of hope for the future.  I believe
> all of these are solvable problems, but I believe we need the will to solve
> them.  Apple’s recent announcement of WebKit2 lockdown is clearly one
> attempt at some of these.  But for the other 50% of WebKit developers who
> don’t work on WebKit2, locking down WebCore is not a good solution.
>
> I think we need to work together to bring about some of these dreams, for
> the short and long term health of the WebKit project.
>
> Thank you for listening.
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>