[webkit-dev] Closing the loop on flaky tests (was Re: Flaky test hit list)

[Alexey Proskuryakov]

19.10.2010, в 11:16, Adam Barth написал(а):

> Also, these bugs are close to the end of their
> lifecycle (because their patch is about to land), so they shouldn't
> generate more than 3 or 4 emails each.  That boils down to about one
> or two emails per week for the flakiest test.


One e-mail (per week?) would perhaps make sense, even though "flaky test" is sometimes "flaky code", so the blame becomes misplaced. Getting 3-4 automated e-mails per bug seems over the board.

I agree that raising awareness of which tests or code areas are flaky seems useful. One problem I personally had was with digging up data on flakiness. The link for a dashboard that I found was <http://test-results.appspot.com/dashboards/flakiness_dashboard.html> - the URL was freezing my browser for several minutes on each move, and I couldn't make sense of what it was telling me UI-wise quickly enough. I'm not even sure how it's related to flakiness seen by commit queue, as it seems to be about chromium.

Is there a better data source that I missed?

- WBR, Alexey Proskuryakov