[webkit-dev] Flaky test hit list
evan at chromium.org
Thu Sep 30 17:33:45 PDT 2010
On Wed, Sep 29, 2010 at 11:31 PM, Adam Barth <abarth at webkit.org> wrote:
> Tonight I wrote a new webkit-patch command for detecting flaky tests.
Are you familiar with the Chromium flakiness dashboard?
It provides a continuous summary of test results aggregated across
multiple builders, including views into the expected/actual results,
timing, logs, etc.
See for example
and click the "show results" link in the lower left to see the failure diff.
The front page, as I understand it, provides a list of the flakiest
tests in order. (It also can cause your browser to melt, be
forewarned.) And the json input the dashboard requires is already
produced by new-run-webkit-tests, so I think it'd be trivial to plug
the other WebKit builders into it. (Given how small your hit list is,
I doubt the WebKit results would cause the dashboard to become as
heavyweight as the Chromium one is.)
PS: along those lines, here's an hierarchical view into test runtime
generated from the results json archived by those bots. I hope to
hook this into the dashboard once I figure a few other things out.
(Copy that URL including the space, sorry, it's a quick hack).
More information about the webkit-dev