[Webkit-unassigned] [Bug 103839] FlakyTestReporter should be re-enabled and taught how to post patches

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=103839





--- Comment #2 from Eric Seidel <eric at webkit.org>  2012-12-02 12:52:01 PST ---
For those who are curious how/why the bot has a hard-coded limit about the number of flakes/failures it can handle while still processing patches, see:
http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/tool/bot/expectedfailures.py#L40
http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/tool/bot/layouttestresultsreader.py#L68
http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/tool/steps/runtests.py#L38
http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/common/net/layouttestresults.py#L152

(I was wrong, the current limit is 30.)

The gist is that we don't want to run all the tests when a patch makes every test crash (that could take hours) so we stop after some number of failures.  If we ever hit that limit, we can no longer trust the results.

Thus if there are 30 failures/flakes then we can't process patches at all.  We just spin until someone fixes the tree.

The proposal in this bug would help keep the number of flakes down, by making it easy for a human (the current webkit sherriff?) to take action to fix the TestExpectations to recognize the flakes.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.