Wow. I have to admit I was skeptical. It's way too early to tell, but I *really* like the pass messages which make my job as a reviewer much easer:<div><div><a href="https://bugs.webkit.org/show_bug.cgi?id=31985#c3">https://bugs.webkit.org/show_bug.cgi?id=31985#c3</a></div>
<div><br></div><div>Thank you very much for putting this together Adam!</div><div><br></div><div>-eric<br><br><div class="gmail_quote">On Mon, Nov 30, 2009 at 4:03 PM, Adam Barth <span dir="ltr"><<a href="mailto:abarth@webkit.org">abarth@webkit.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">The queue is now running and caught up to the top of the review queue.<br>
Please let me know if we should adjust anything.<br>
<br>
Thanks,<br>
<font color="#888888">Adam<br>
</font><div><div></div><div class="h5"><br>
<br>
On Sat, Nov 28, 2009 at 2:21 AM, Adam Barth <<a href="mailto:abarth@webkit.org">abarth@webkit.org</a>> wrote:<br>
> One of the bots that Eric and I have been working on is about to come<br>
> online. This bot is a "style bot" that runs check-webkit-style on<br>
> patches that have been nominated for review. I'd like to ask your<br>
> patience while we work out the kinks.<br>
><br>
> You can skip the rest of this email if you're not interested in the<br>
> nitty gritty.<br>
><br>
> == Summary ==<br>
><br>
> The primary goal of the style-queue is to reduce review latency by (1)<br>
> giving immediate feedback to contributors and (2) making human<br>
> reviewer more efficient by relieving them of mechanical tasks (like<br>
> asking for tabs to be replaced with spaces).<br>
><br>
> The style-queue scans the set of patches that are pending-review for<br>
> new patches. When it finds a new patch, the bot runs<br>
> check-webkit-style on the patch. If check-webkit-style finds style<br>
> errors, the bot comments on the bug.<br>
><br>
> == Social Contract ==<br>
><br>
> The style-queue is purely informative. You're free to ignore it's<br>
> comments. Ideally, the bot should be silent unless it has something<br>
> interesting to say. If we decide that style-queue is too chatty, we<br>
> can change check-webkit-style to report fewer errors.<br>
><br>
> One corollary of this social contract is that the style-queue doesn't<br>
> inform you when your patch passes the style check (because most<br>
> patches should pass, right?). We can re-visit this design choice<br>
> later if it turns out we really want to know when the check succeeds.<br>
><br>
> == Response to Previous Feedback ==<br>
><br>
> When I floated this idea on webkit-dev previously (in the "bot-filled<br>
> future" thread), I got a bunch of useful feedback. Here's how we<br>
> incorporated that feedback into the current design:<br>
><br>
> 1) "Adding an extra flags is going to cause confusion." The<br>
> style-queue does not add any flags to Bugzilla. Instead of storing<br>
> it's state in Bugzilla flags (like commit-queue does), we built an<br>
> AppEngine web service to hold the queue's persistent state. Instead<br>
> of indicating style errors with a negative flag, the bot adds a<br>
> comment to the bug.<br>
><br>
> 2) "What machines are going to be doing these tests, and on which<br>
> platforms?" The style-queue runs on a machine in my apartment that<br>
> runs Mac OS X. The style-queue is platform independent, so this is<br>
> less of an issue for style-queue than it is for a build-queue.<br>
><br>
> 3) "Which patches would this test?" The style-queue tests any patch<br>
> marked "review?". This design avoids the need for additional flags or<br>
> indications that a patch should be run through the queue.<br>
><br>
> 4) "Running tests on an arbitrary patches [opens a security hole]."<br>
> Instead of running check-webkit-style from it's own working copy, the<br>
> style-queue runs check-webkit-style from a separate working copy.<br>
> While it's true you could haxor my machine by committing an evil copy<br>
> of check-webkit-style, I'm already running that risk whenever I type<br>
> "svn up".<br>
><br>
> == Frequently Asked Questions ==<br>
><br>
> Q1: If the style-queue doesn't complain, does that mean my patch has<br>
> correct style?<br>
><br>
> A1: Unfortunately, no. First of all, check-webkit-style has false<br>
> negatives. Hopefully, the script will improve over time, but it will<br>
> never be perfect. Second, the style-queue is only able to check<br>
> patches that successfully download and apply to top-of-tree. If your<br>
> patch does not apply to top-of-tree (e.g., because it depends on<br>
> another patch), then style-queue won't say anything.<br>
><br>
> Q2: How can I see whether style-queue processed my patch?<br>
><br>
> A2: There isn't a great way to do this yet. The data is stored in the<br>
> <a href="http://webkit-commit-queue.appspot.com" target="_blank">webkit-commit-queue.appspot.com</a> web service, but we haven't built a<br>
> front-end for it yet. If you have a strong stomach, you can look at<br>
> the endpoint the bot itself uses<br>
><br>
> <a href="http://webkit-commit-queue.appspot.com/patch-status/style-queue/12345" target="_blank">http://webkit-commit-queue.appspot.com/patch-status/style-queue/12345</a><br>
><br>
> where 12345 is the ID of your attachment. If you'd like to have a<br>
> better status display, feel free to hack on<br>
> WebKitTools/QueueStatusServer and send me or Eric patches to review.<br>
> In the long term, there's no reason we can't have an awesome UI.<br>
><br>
> Q3: This bot is going to run amock! How can I keep track of all it's<br>
> evil deeds?<br>
><br>
> A3: If you want to see what the bots are doing, you can subscribe to<br>
> the following email list:<br>
><br>
> <a href="http://groups.google.com/group/webkit-bot-watchers" target="_blank">http://groups.google.com/group/webkit-bot-watchers</a><br>
><br>
> Whenever the bots touch a bug, they're supposed to add this address to<br>
> the CC list. This functionality is very new, so we might not have<br>
> every case covered yet. If you see the bot misbehaving, let me or<br>
> Eric know. We'll correct the issue ASAP.<br>
><br>
> Q4: Checking style is fine and dandy, but I was hoping for a build-queue!<br>
><br>
> A4: My next project is to get a build-queue working for Qt. The<br>
> build-queue will share a lot of infrastructure with the style-queue,<br>
> but making the style-queue was easier because check-webkit-style is a<br>
> lot faster than build-webkit. :)<br>
><br>
> Q5: When are you flipping the switch?<br>
><br>
> A5: Either today or tomorrow, depending on when I find a stretch of<br>
> time to babysit the queue through whatever initial troubles it has.<br>
> My working copy of the style-queue is about five patches ahead of<br>
> trunk. It would be nice to have those patches committed before<br>
> flipping the switch, but it's not necessary.<br>
><br>
> Thanks for reading to the end, and I hope you find the style-queue useful.<br>
><br>
> Adam<br>
><br>
</div></div></blockquote></div><br></div></div>