[webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

Tue Dec 3 21:31:47 PST 2019

Great. I've completed the survey.

- R. Niwa

On Mon, Dec 2, 2019 at 5:19 PM Aakash Jain <aakash_jain at apple.com> wrote:

> There were multiple ideas discussed in this thread. I would like to gather
> more data about what do most people prefer. I have sent out a short survey
> in https://lists.webkit.org/pipermail/webkit-dev/2019-December/030980.html
>
> Thanks
> Aakash
>
> On Nov 5, 2019, at 12:04 PM, Alexey Proskuryakov <ap at webkit.org> wrote:
>
>
>
> 4 нояб. 2019 г., в 1:37 PM, Ryosuke Niwa <rniwa at webkit.org> написал(а):
>
>
> On Mon, Nov 4, 2019 at 9:40 AM Alexey Proskuryakov <ap at webkit.org> wrote:
>
>>
>> Can you elaborate on that, how exactly is e-mailing on first failure
>> useful to reviewers?
>>
>> Getting rid of Bugzilla comments was one of the goals of EWS rewrite,
>> based on engineering feedback about noise in bugs and in e-mail, and I
>> wholeheartedly agree with this feedback. So I think that comments are
>> generally undesirable.
>>
>> Since I don't understand what your precise scenario is, I may be make
>> straw man arguments below, but here are some things that I think make the
>> proposed behavior unhelpful (add a comment on first failure, or when all
>> EWSes pass).
>>
>> 1. EWS comments in Bugzilla are so annoying that some people take the
>> radical step of manually hiding them. EWS history is archived anyway, there
>> is no need to look into comments for it.
>>
>> 2. There are often many people CC'ed on the bug to whom EWS data is
>> irrelevant or even mysterious (e.g. reporters, web developers or
>> non-reviewers). The noise is a slight annoyance, discouraging further
>> participation in the project.
>>
>> 3. I believe that for most reviewers, the mode of operation is one of the
>> two: (1) do it when pinged directly, or (2) go over the review queue when
>> one has the time. Getting EWS comments helps neither.
>>
>> 4. Commenting when all EWSes pass is not very practical - it's too often
>> that we have some stragglers that take days (or forever). I don't think
>> that we can make it reliable even if we start actively policing EWS
>> responsiveness.
>>
>> 5. The reviewer likely wants to know the state of multiple EWSes if they
>> are going to wait for EWS at all. What exactly are they going to do after
>> getting an e-mail that one EWS failed?
>>
>
> I often use a EWS failure as a signal to wait reviewing a patch.
> Otherwise, a bug mail will stay in my inbox as one of items to get to.
>
> I can see the usefulness in the somewhat unusual case of a super urgent
>> patch. We may want multiple people to watch it, so that members of CC list
>> would go and ask the patch author to update it with more urgency than
>> e-mail allows for. I think that opt-in is a better mechanism for that, so
>> that people who opted in would receive information about each EWS data
>> point.
>>
>
> I think there is a value in knowing that a patch isn't ready instead of
> having to open the bug to realize that.
>
>
> So just to clarify,
>
> - a major part of how you get to review bugs is by being CC'ed, and you
> review them when you have the time to read bugmail;
> - and you don't open the bug in Bugzilla if there is already an EWS
> failure by the time you read the e-mail where review is requested?
>
> That's clearly a valid benefit. In my mind, it probably doesn't outweigh
> the downsides. On the other hand, yours is a voice of someone who reviews
> way more patches than Maciej and me combined these days, so maybe more
> e-mail is an overall benefit to many of the reviewers.
>
> - Alexey
>
>
>
> - R. Niwa
>
>> 3 нояб. 2019 г., в 6:58 PM, Maciej Stachowiak <mjs at apple.com> написал(а):
>>
>>
>> I think they are useful to actual and potential reviewers. Direct email
>> to the patch author is not something anyone can Cc themselves on, and is
>> not archived, so seems like a strictly worse form of communication.
>>
>> On Nov 2, 2019, at 9:34 AM, Alexey Proskuryakov <ap at apple.com> wrote:
>>
>>
>> My preference is still e-mailing the patch author directly (possibly,
>> also having an option to opt in for anyone). Bugzilla comments will always
>> be irrelevant for most people CC'ed on the bug, and they are almost always
>> undesirable to keep within the discussion flow.
>>
>> - Alexey
>>
>> 1 нояб. 2019 г., в 18:28, Aakash Jain <aakash_jain at apple.com> написал(а):
>>
>> Sounds good. I prefer the single comment when the first failure occur.
>> That way notification would be sent as soon as the first failure happens.
>>
>> I'll implement that (assuming it's acceptable to everyone).
>>
>> Thanks
>> Aakash
>>
>> On Nov 1, 2019, at 8:35 PM, Maciej Stachowiak <mjs at apple.com> wrote:
>>
>>
>> How about only a single comment when the first failure occurs? (Or else
>> when all bots pass, if there is never a failure.)
>>
>> This should help the author, the reviewer, and anyone else cc’d, without
>> being too spammy.
>>
>> On Nov 1, 2019, at 5:20 PM, Aakash Jain <aakash_jain at apple.com> wrote:
>>
>> Hi Ryosuke,
>>
>> Many people didn't like the noise by the EWS comments, and we removed the
>> comments as per previous discussion in:
>> https://lists.webkit.org/pipermail/webkit-dev/2019-June/030683.html.
>>
>> I agree with your point that having some kind of notification might be
>> useful.
>>
>> I proposed some ideas in
>> https://lists.webkit.org/pipermail/webkit-dev/2019-September/030798.html,
>> but didn't get much feedback. If we can all agree on a solution, I can look
>> into implementing it.
>>
>> Thanks
>> Aakash
>>
>> On Oct 30, 2019, at 1:03 AM,
>> - R. Niwa
>> <rniwa at webkit.org> wrote:
>>
>> These enhancements are great. There is one feature of the old EWS that I
>> really miss, which is that I used to get emails when some EWS failed. With
>> new EWS, I have to keep checking back the bugzilla to see if any of them
>> have failed periodically.
>>
>> Can we add a feature to opt into such an email notification? Maybe a flag
>> on a patch or JSON configuration file somewhere.
>>
>> - R. Niwa
>>
>> On Tue, Oct 29, 2019 at 4:05 PM Aakash Jain <aakash_jain at apple.com>
>> wrote:
>> Hi Everyone,
>>
>> I am happy to announce another EWS feature.
>>
>> From now on, in case of build failure, EWS will parse the errors and
>> display them in a separate 'errors' log. You wouldn't have to search
>> through thousands of lines of logs to find the error message.
>>
>> For example, in https://ews-build.webkit.org/#/builders/16/builds/6054,
>> in step #7 WebKit failed to compile. Complete logs (stdio) are 38,000+
>> lines, and the error is not at the end of the logs. Normally, it requires
>> some searching through the logs to find the relevant errors. But now, there
>> is another 'errors' log, which contains just the relevant 11 lines
>> (containing error and few related lines to provide additional context).
>>
>> Hopefully this would save some time and efforts previously spent on
>> searching through the large logs.
>>
>> Note that this information is not displayed in status-bubble tool-tip,
>> since this might be lot of text to display in the tooltip. My further plan
>> is to make this information more readily available, by adding it to a
>> custom designed page which will open on clicking the status bubble
>> https://webkit.org/b/197522
>>
>> Please let me know if you notice any issues or have any feedback.
>>
>> Thanks
>> Aakash
>>
>> Reference: https://webkit.org/b/203418
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>> --
>> - R. Niwa
>>
>>
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>>
>>
>>
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>>
>>
>> - Alexey
>>
>>
>>
>>
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>>
>
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20191203/1c9e0918/attachment.htm>