[webkit-dev] Best practices for failing a flaky tests (was Re: Switching to new-run-webkit-tests)

[Ryosuke Niwa]

On Wed, Jul 6, 2011 at 9:00 AM, Adam Roben <aroben at apple.com> wrote:
>
> OK. Then I have another question:
>
> What should I do to make the Leopard and SnowLeopard bots green, now that
> they have switched to NRWT?
>

Looking at
http://build.webkit.org/results/SnowLeopard%20Intel%20Release%20(Tests)/r90458%20(31133)/results.html

You can http/tests/cookies/third-party-cookie-relaxing.html
and storage/domstorage/localstorage/storagetracker/storage-tracker-7-usage.html
are real failures so you can add skip those, rebaseline, or revet changes
that caused the failure.

For flaky tests, you can add
BUG##### : http/tests/misc/favicon-loads-with-icon-loading-override.html =
TEXT PASS
in mac or mac-leopard test_expectations.txt

Although flaky tests only make the bots orange.

- Ryosuke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20110706/47d062d5/attachment.html>