[webkit-dev] Issue reports, merge requests, etc. (was: WebKit Transition to Git)

[Ryosuke Niwa]

I feel like I should write longer replies but here we go.

On Fri, Oct 2, 2020 at 12:53 PM Michael Catanzaro <mcatanzaro at gnome.org> wrote:
>
> On Fri, Oct 2, 2020 at 11:51 am, Ryosuke Niwa <rniwa at webkit.org> wrote:
> > Since Igalia has a lot
> > more experience working with other open source projects, do you have
> > some suggestions in how to approach that?
>
> Sorry for a long-ish mail. I didn't mean for this to turn into an
> essay. Too late. :P
>
> I actually moved to Red Hat, but anyway, I would say first step is to
> ensure every incoming issue is triaged, at least once when initially
> reported, and every merge request given at least cursory review within
> a reasonable amount of time. That's how open source projects attract
> and retain community contributors, and it's been a weakness for WebKit
> for a while (we're not the worst at this, but certainly not good at it
> either). One possible answer for this is to have a "project manager" or
> similar role to triage and prioritize issues, but of course it can also
> be done by developers working together (possibly according to some
> rotation).
>
> WebKit is such a big project that I suspect almost nobody has enough
> expertise to triage all incoming bugs, but most of us have enough
> expertise to figure out which issue labels to apply to an incoming bug
> to pass the issue off to more specialized experts. On GNOME GitLab we
> label incoming issues with Bug, Crash, Enhancement, or Feature (the
> difference between Enhancement and Feature is hazy) and have a variety
> of global and per-project labels that can also be applied to issues
> [1]. The list of current components in WebKit Bugzilla would be a good
> starting point to use here. Every new issue gets reviewed and either
> gets closed or else gets a label applied to indicate it has been
> triaged, usually within one business day. If an issue is open and
> doesn't have any labels, we know it has not been triaged.
>
> A lot of bugs can be closed immediately because they're reported in the
> wrong place, for instance. Most crashes are reported without a
> backtrace; to those, we attach a Needs Information label and point the
> reporter to instructions for getting a backtrace with gdb, and close
> after a few days if not provided. The details don't matter so much as
> the fact that the issue has received minimal human attention. In the
> rare cases where a bug report is perfect and all I need to do is add
> issue labels, I give the issue an upvote to indicate I've seen it, so
> the reporter knows it hasn't been *completely* ignored, even if nobody
> fixes it. The details don't matter so much as that contributors and
> issue reporters feel like they've received at least some attention and
> aren't shouting into a bug wasteland.

Yeah, good bug triaging seems to be a key. A lot of people complain
about how we don't fix bugs they filed as well.

> Then developers subscribe to project-specific labels in accordance with
> their expertise and tolerance for mail. E.g. I watch all issues for
> some projects, but for GLib I'm only subscribed to networking-related
> labels, since problems in the network-related code often impact WebKit.
> Some developers will want to watch only for CSS bugs, only layout bugs,
> only JSC bugs, only WebKitGTK bugs, etc.
>
> Of course, triaging issues doesn't mean they actually get fixed, but
> it's a necessary first step. (And once isn't really enough, either.
> Most of our bugs from 2008 are probably no longer valid, but it takes
> time to review old bugs, and not many people have enough specialized
> technical expertise to triage outside their area of expertise. Anyway,
> once is a starting point.)

Right,  we'd probably continue to receive more bugs than we can chew
on so we'd need some process for prioritization as well.

> Responding to merge requests (or patch reviews) in a timely manner is
> also important (otherwise, contributors disappear). Merge requests take
> longer than issues, but it's good to get to each one within a couple
> days; otherwise, they really start to pile up. We currently have
> patches awaiting review from 2017 [2], and that's just the last time
> somebody decided to mass-deny old review requests. And a lot of these
> are from Apple/Igalia/Sony, who all know who to CC for reviews; imagine
> how much harder it is for someone not familiar with the WebKit
> community to get a patch reviewed. ;) GitHub will probably help with
> this because it puts merge requests front-and-center in its UI, instead
> of requiring us to first discover then take the time to sort through
> the request queue. It's hard to use GitHub without habitually reviewing
> pending requests. ;) The number of open issues and MRs is even clearly
> displayed as a sort of challenge to reduce the number. But we'd need
> some workflow for directing merge requests to appropriate reviewers.
>
> When we move to GitHub, you can expect to start receiving merge
> requests from people who would not take the time and effort to create a
> WebKit Bugzilla account and learn how to use our arcane ChangeLog and
> patch creation scripts. Epiphany has twice as many active contributors
> and probably 3-4x more activity today on GNOME GitLab than it did when
> we used GNOME Bugzilla, and I'm pretty sure that's mainly due to GitLab
> rather than other factors. I wouldn't expect that dramatic a change for
> WebKit, since most WebKit contributors are professional rather than
> volunteer contributors, but I'd guess we'll attract at least that much
> more volunteer community contributor activity. It might take a year or
> two for the change to really become noticeable, but I expect it will be
> significant because moving to GitHub eliminates several barriers to
> contribution.

I'm actually a bit concerned about the amount of noise & spam we'd get here.
In various W3C repos I'm active, we do get a fair bit of spam /
non-sensical issues & PRs.

> One more thing we can do is to stop updating the ChangeLog files. These
> files are archaic and not very useful, as long as we continue to write
> good, descriptive commit messages.

Well, that's a big topic on its own. The biggest issue I see is that
GitHub's code review tool doesn't let you add inline comments to
commit messages.

> Lastly, I'll just say that GitHub CI should be a lot nicer than EWS
> bubbles in Bugzilla. ;) I don't have any experience with GitHub CI -- I
> hear it's not quite as powerful as GitLab's? -- but I assume it allows
> us to upload podman/docker images to run whatever checks we want. This
> is really great and the more we take advantage of it, the better.

WPT uses it and it's okay. One annoying thing is that CI can't really
surface a lot of information to GitHub.
FWIW, you can probably get away with builders but testers would behave
differently if you're running in VM vs. real hardware in terms of
graphics stack, etc... so I'm not sure if you want to do that.

- R. Niwa