[webkit-dev] Closing the loop on flaky tests (was Re: Flaky test hit list)

[Ojan Vafai]

On Tue, Oct 19, 2010 at 11:44 AM, Alexey Proskuryakov <ap at webkit.org> wrote:

> I agree that raising awareness of which tests or code areas are flaky seems
> useful. One problem I personally had was with digging up data on flakiness.
> The link for a dashboard that I found was <
> http://test-results.appspot.com/dashboards/flakiness_dashboard.html> - the
> URL was freezing my browser for several minutes on each move, and I couldn't
> make sense of what it was telling me UI-wise quickly enough. I'm not even
> sure how it's related to flakiness seen by commit queue, as it seems to be
> about chromium.


That dashboard currently only supports the Chromium bots. If other bots
successfully switch over to new-run-webkit-tests, we'll be able to easily
add them to that dashboard.

The freezing issue is a recent one I plan on looking into soon. WebKit is
ridiculously slow at rendering this HTML for some reason (it's a single
large table).

The UI is very dense and confusing, but it gives you quite a bit of useful
information. Here's some limited documentation on making sense of the
dashboard UI:
http://sites.google.com/a/chromium.org/dev/developers/testing/flakiness-dashboard

Ojan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20101019/cffc0c35/attachment.html>