[webkit-dev] Open Design beyond Open Source

[Lars Knoll]

Hi George and Maciej,

let me just add in some comments from my side :)

On Saturday 17 February 2007 08:23, Maciej Stachowiak wrote:
> Hey George,
>
> Thanks for sharing your concerns.
>
> On Feb 16, 2007, at 10:29 PM, George Staikos wrote:
> > On 16-Feb-07, at 9:22 PM, Mike Emmel wrote:
> >> Here is the link of what you have to go through to commit now.
> >>
> >> http://webkit.org/coding/contributing.html
> >>
> >> People wanting to do a new port are not going to go through this long
> >> trial program just to commit in fact that have not.  The KDE team has
> >> pre-existing interest and both OSX and S60 are commercial.  The  two
> >> true open source ports gdk/windows plus the wx widget port have
> >> basically been failures so far.
> >
> >    If you mean to say that the impression given off by the webkit
> > project is one of "we don't want you" or at least "we're not really
> > interested in having more people join our project", I have to
> > agree.  I still don't get the feeling of a welcoming project.   Is
> > it a form of extreme paranoia that some new developer might
> > introduce a bug?  Maybe.... I'm not sure I consider that a valid
> > reason though.
>
> We really do want the WebKit project to be open to new contributors.
> We spend a lot of effort working on this. If we didn't care about
> having new developers join, we wouldn't have the whole public
> repository and web site at all.

Fortunately this has happened. We've had a split situation (between apple and 
KDE) for years, and with the WebKit project, we can finally start to pull 
into the same direction. Not everything is perfect currently (as some of the 
mails show), but in general I see things moving into the right direction.

> It is true that the WebKit project has somewhat higher barriers to
> entry than other large open source projects, like KDE. But I believe
> the barriers are lower than some others, like Mozilla.

Well, an HTML rendering engine has basically by definition a higher level of 
entrance as a project as KDE. This is purely due to the fact that the 
problems are more complex. With a huge desktop as KDE, people can always find 
corners that suit them and are easy enough to deal with. To be able to 
contribute in a good way to WebKit (at least code wise) requires some more 
knowledge to start out with.

Also in KDE the number of contributors to khtml was always rather small, and 
Dirk, Harri, Antti and myself kept a rather close eye on the core parts (DOM, 
CSS, JS and rendering). Also there every commit was reviewed. This often 
happened after the fact, which gave us problems and long discussions from 
time to time. So a pre submit review is IMO a good thing.

> Here are the barriers I think we have:
>
> 1) We require code review for all changes.
>
> 2) We do not grant commit access automatically; we expect people to
> have a track record of being good contributors. Note that this is
> more about showing that you can work with others and are not a source
> of conflict or negativity. It is not about coding skills.
>
> 3) We do not give review privileges automatically. This is more about
> understanding project policies and having good judgment
>
> 4) We require all code changes that affect processing of web content
> to come with a regression test case (if it is possible to make one).
>
> I think getting a patch reviewed is easier than in Mozilla, where you
> need a review and a "superreview" and adherence to all review
> happening in bugzilla is much more strict (we often bypass that and
> do review via email or IRC for established developers). But perhaps
> it is harder than for KDE, where things are often committed without
> review.

I agree to both here. Having worked a bit with mozilla as well, I have to say 
that WebKit is definively doing a lot better.

> I think it is also easier to get commit privileges than in Mozilla. I
> checked some stats, and everyone who has contributed more than 5
> patches so far this year already has commit rights. That says to me
> that the people actively contributing have the access they need. Now,
> it may be that some people were discouraged from even contributing in
> the first place. But that shows a lack of commitment to me.
>
> >> So from and opens source perspective webkit is not a smashing success
> >> in drawing in the open source community only the KDE team has really
> >> contributed and they were the original developers. The S60 port also
> >> has a history of working with KHTML before webkit was made
> >> a open project.
> >
> >   I wouldn't say so.  I think the KDE contribution is effectively
> > nil.  The Qt port is distinct from KDE plans.  KDE as a community
> > has yet to find a way to work together with WebKit for reasons that
> > I think you're also experiencing and perhaps not clearly
> > articulating.  Yes I'm writing this from @kde.org and have
> > contributed a huge amount to KDE, KHTML, etc over the past 8-9
> > years.  Until I have the KDE community feeling welcome and
> > confident in participating, and until we have functioning code and
> > development happening for a useful kpart, I don't see our Qt port
> > as anything relevant to KDE.  I'm trying to make it relevant, and I
> > want it to be relevant, but it really isn't.

This is one of the remaining issues that require solving. Currently the work 
on the Qt port is mostly done by a few people coming from the KDE community 
and Trolltech, but we unfortunately still can't say that it's really 
supported by the community.

IMO the problems stem mostly from past history and not so much from what 
WebKit currently is. But there is still work that needs to be done on both 
sides to finally get to a point where we can talk about a real cooperation.

> >    I've had a webkit SVN account for almost 2 years now.  I first
> > tried to merge WebKit code back to KDE after all the KDE developers
> > effectively gave up, and after successfully merging JavaScriptCore
> > (with Maksim's help), determined that it just wasn't feasible to go
> > any further without a complete replacement.  I then started the
> > current Qt port and managed to enlist several KDE developers to
> > help me.  At least one has given up again already, unsurprisingly.
> > We made some great progress but it's an absolutely ridiculous
> > development model as far as getting real work done.  So far we had:
> >
> > 1) Work outside webkit SVN to start the port
> >    -> webkit renames and refactors hit us hard
> > 2) Port again, also outside of WebKit SVN (note: we were doing a
> > very large number of commits/week.  far more than now)
> > 3) Spend far too much time merging from WebKit SVN, especially when
> > things like renames and refactors happen
> > 4) Eventually give up and merge back to webkit SVN

I wouldn't see it quite as negative. Yes we had some larger problems. Some of 
them stem from the fact that we were working on a private repository for a 
while, some from the huge refactorings that happened in WebKit in Oct-Dec. 
The private repository is gone, we're now working on ToT and most large 
refactorings are finished, so I think this was more a one time thing and will 
not repeat itself.

> > 5) End up in a ridiculous situation of having one member of the
> > porting team as the only person with rights to review the work.
> > Interesting, since that person has no-one who can review his work
> > under the formal rules.  The guy who started the port, and the guy
> > who invented KHTML altogether are not given review rights.
> > 6) Work slows down drastically as developers are discouraged and
> > are stuck in procedure that has relatively little value for the
> > given situation.
> >
> >    Not a good model.  Maybe it works in an office with a couple of
> > infrequent contributors, but it doesn't work so well for a
> > distributed network of people trying to contribute significant
> > amounts of code.  I fully agree that some sections of webkit need
> > strict review control, testcases, bug reports, and other formal
> > procedures.  I strongly support it in fact.  The current situation
> > is not limited to just this.  Moreover, it feels very much like a
> > Brusselized project, in stark contrast to where it came from
> > originally.  This feeling applies to "ports" as much as anything
> > else right now.
>
> I think you are right that the process may be overkill for new ports
> that are still under construction. In the early stages, you want to
> move quickly, not be overly bogged down by process. Here are some
> thoughts I've had on making it better:

Absolutely. Having some dirty edges here and there or getting a regression 
from time to time doesn't matter as much as it would for a well worked out 
and stable port. Being able to move fast is a lot more important. As a port 
matures this will of course change.

> * We offer ports the opportunity to have their own branch where they
> can have whatever review policy they like. We should probably do
> something similar for the port-specific code for ports that are
> living on the trunk. Either let such ports make up their own review
> policy for purely port-specific code, or give many more people review
> rights for at least the purely port-specific code.

I don't think a branch is a good idea. I am very happy that we are working on 
trunk, otherwise merging stuff back to trunk later on would have been way too 
painful.

> (On the other hand, when port code isn't reviewed at all by people
> working on other platforms, it makes it more difficult to maintain
> coding standards, share knowledge, and ensure that core changes don't
> break ports, so some degree of cross-review is probably helpful.)
>
> * I'm happy to give you and Lars review rights for Qt/KDE platform-
> specific code, right now if you want it. We don't usually talk about
> this kind of stuff in public, but since you raised it in public, we
> would definitely give Lars general review rights as soon as he is
> sufficiently familiar with our particular version the core code.
>
> * We would give more KDE developers commit rights, but so far no one
> else has asked, or started submitting patches. I don't know what to
> do about it for other ports. For what it's worth: currently 6 KDE/Qt
> developers have commit rights, and we made the offer to one more who
> declined at the time.

To get them, they have to feel welcome and get the feeling they are heard and 
respected. A lot of them still think that WebKit is a project controlled by 
Apple, and that's not what they're looking for. 

> * Refactoring/renaming breaking code is a hassle, but (a) it becomes
> less of an issue for more mature ports that are living on the trunk
> and have a BuildBot set up; and (b) a lot of the heavy refactoring
> will be very good for ports in the long run, since it moved a lot of
> code down from Cocoa/ObjC to platform-neutral C++ code. Other than
> that, I don't know what to do about it. It's hard to track a fast-
> changing project.

I don't think this is an issue. The refactorings sometimes hurt in the short 
term, but in the long term, they'll help us all. One thing I would wish for 
though is more open discussions (on e.g. this mailing list) about 
architectural changes before they actually happen. 

I would also hope that we can get better in communicating and discussing 
planning in public. Things like when a freeze is supposed to happen should be 
discussed and agreed on in public (and announced in good time ahead), and we 
should ideally have something out on in the public that allows everyone to 
align it's plans to it.

And: IRC is IMO a good way to get some initial opinions, but only using IRC 
leaves people out. Partly due to time zone differences, partly because some 
people are just not always online and might miss the particular discussion.

> If you have any other specific suggestions for I would be glad to
> hear them. I feel bad that you are frustrated, and we will gladly
> listen to constructive suggestions. I can't promise we will take all
> suggestions but we will give them consideration. You also know my
> phone number and my email if you'd like to talk to me personally,
> which I am happy to do any time.

Which gives rise to the question as to who you mean by 'we'. The project 
itself is open, but what about the way it is steered?

> >> In my book this makes WebKit a failed open source project  so far.
> >
> >    I wouldn't go that far.  I would call it a successful open
> > source project with a rather closed community at the moment.  Open
> > source doesn't need community, but it sure helps.  I really don't
> > think there are any technical problems, just social ones.
>
> FYI, three of the top 5 patch submitters for 2007 so far are not
> Apple employees (mitzpettel, ap, lars). So I don't think the process
> stops non-Apple coders from participating. But it's true that most of
> the code changes being made are still by Apple employees, if only
> because a lot of us are working on it full time.

It certainly didn't fail. The interest we see from all sides in the project 
clearly shows the opposite. But as this thread shows there are issues that 
need to be solved (or clarified).

Cheers,
Lars