[webkit-dev] EWS Comments on Bugzilla (Was: EWS now parses error logs in case of build failure)

Mon Nov 4 13:37:02 PST 2019

On Mon, Nov 4, 2019 at 9:40 AM Alexey Proskuryakov <ap at webkit.org> wrote:

>
> Can you elaborate on that, how exactly is e-mailing on first failure
> useful to reviewers?
>
> Getting rid of Bugzilla comments was one of the goals of EWS rewrite,
> based on engineering feedback about noise in bugs and in e-mail, and I
> wholeheartedly agree with this feedback. So I think that comments are
> generally undesirable.
>
> Since I don't understand what your precise scenario is, I may be make
> straw man arguments below, but here are some things that I think make the
> proposed behavior unhelpful (add a comment on first failure, or when all
> EWSes pass).
>
> 1. EWS comments in Bugzilla are so annoying that some people take the
> radical step of manually hiding them. EWS history is archived anyway, there
> is no need to look into comments for it.
>
> 2. There are often many people CC'ed on the bug to whom EWS data is
> irrelevant or even mysterious (e.g. reporters, web developers or
> non-reviewers). The noise is a slight annoyance, discouraging further
> participation in the project.
>
> 3. I believe that for most reviewers, the mode of operation is one of the
> two: (1) do it when pinged directly, or (2) go over the review queue when
> one has the time. Getting EWS comments helps neither.
>
> 4. Commenting when all EWSes pass is not very practical - it's too often
> that we have some stragglers that take days (or forever). I don't think
> that we can make it reliable even if we start actively policing EWS
> responsiveness.
>
> 5. The reviewer likely wants to know the state of multiple EWSes if they
> are going to wait for EWS at all. What exactly are they going to do after
> getting an e-mail that one EWS failed?
>

I often use a EWS failure as a signal to wait reviewing a patch. Otherwise,
a bug mail will stay in my inbox as one of items to get to.

I can see the usefulness in the somewhat unusual case of a super urgent
> patch. We may want multiple people to watch it, so that members of CC list
> would go and ask the patch author to update it with more urgency than
> e-mail allows for. I think that opt-in is a better mechanism for that, so
> that people who opted in would receive information about each EWS data
> point.
>

I think there is a value in knowing that a patch isn't ready instead of
having to open the bug to realize that.

- R. Niwa

> 3 нояб. 2019 г., в 6:58 PM, Maciej Stachowiak <mjs at apple.com> написал(а):
>
>
> I think they are useful to actual and potential reviewers. Direct email to
> the patch author is not something anyone can Cc themselves on, and is not
> archived, so seems like a strictly worse form of communication.
>
> On Nov 2, 2019, at 9:34 AM, Alexey Proskuryakov <ap at apple.com> wrote:
>
>
> My preference is still e-mailing the patch author directly (possibly, also
> having an option to opt in for anyone). Bugzilla comments will always be
> irrelevant for most people CC'ed on the bug, and they are almost always
> undesirable to keep within the discussion flow.
>
> - Alexey
>
> 1 нояб. 2019 г., в 18:28, Aakash Jain <aakash_jain at apple.com> написал(а):
>
> Sounds good. I prefer the single comment when the first failure occur.
> That way notification would be sent as soon as the first failure happens.
>
> I'll implement that (assuming it's acceptable to everyone).
>
> Thanks
> Aakash
>
> On Nov 1, 2019, at 8:35 PM, Maciej Stachowiak <mjs at apple.com> wrote:
>
>
> How about only a single comment when the first failure occurs? (Or else
> when all bots pass, if there is never a failure.)
>
> This should help the author, the reviewer, and anyone else cc’d, without
> being too spammy.
>
> On Nov 1, 2019, at 5:20 PM, Aakash Jain <aakash_jain at apple.com> wrote:
>
> Hi Ryosuke,
>
> Many people didn't like the noise by the EWS comments, and we removed the
> comments as per previous discussion in:
> https://lists.webkit.org/pipermail/webkit-dev/2019-June/030683.html.
>
> I agree with your point that having some kind of notification might be
> useful.
>
> I proposed some ideas in
> https://lists.webkit.org/pipermail/webkit-dev/2019-September/030798.html,
> but didn't get much feedback. If we can all agree on a solution, I can look
> into implementing it.
>
> Thanks
> Aakash
>
> On Oct 30, 2019, at 1:03 AM,
> - R. Niwa
> <rniwa at webkit.org> wrote:
>
> These enhancements are great. There is one feature of the old EWS that I
> really miss, which is that I used to get emails when some EWS failed. With
> new EWS, I have to keep checking back the bugzilla to see if any of them
> have failed periodically.
>
> Can we add a feature to opt into such an email notification? Maybe a flag
> on a patch or JSON configuration file somewhere.
>
> - R. Niwa
>
> On Tue, Oct 29, 2019 at 4:05 PM Aakash Jain <aakash_jain at apple.com> wrote:
> Hi Everyone,
>
> I am happy to announce another EWS feature.
>
> From now on, in case of build failure, EWS will parse the errors and
> display them in a separate 'errors' log. You wouldn't have to search
> through thousands of lines of logs to find the error message.
>
> For example, in https://ews-build.webkit.org/#/builders/16/builds/6054,
> in step #7 WebKit failed to compile. Complete logs (stdio) are 38,000+
> lines, and the error is not at the end of the logs. Normally, it requires
> some searching through the logs to find the relevant errors. But now, there
> is another 'errors' log, which contains just the relevant 11 lines
> (containing error and few related lines to provide additional context).
>
> Hopefully this would save some time and efforts previously spent on
> searching through the large logs.
>
> Note that this information is not displayed in status-bubble tool-tip,
> since this might be lot of text to display in the tooltip. My further plan
> is to make this information more readily available, by adding it to a
> custom designed page which will open on clicking the status bubble
> https://webkit.org/b/197522
>
> Please let me know if you notice any issues or have any feedback.
>
> Thanks
> Aakash
>
> Reference: https://webkit.org/b/203418
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
> --
> - R. Niwa
>
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
> - Alexey
>
>
>
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20191104/c89018f6/attachment.htm>