[Webkit-unassigned] [Bug 61061] TestFailures page blames arbitrary revisions for breaking flaky tests

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=61061


Joseph Pecoraro <joepeck at webkit.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |joepeck at webkit.org




--- Comment #5 from Joseph Pecoraro <joepeck at webkit.org>  2011-06-29 12:49:34 PST ---
View in context: https://bugs.webkit.org/attachment.cgi?id=99100&action=review

> Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/FlakyLayoutTestDetector.js:35
> +            return false;

Since the expected output of this function is an Array, it might make more sense to return null (still falsey).

> Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/LayoutTestHistoryAnalyzer.js:108
> +            var newFlakyTests = self._flakinessDetector.incorporateTestResults(nextBuildName, tests, tooManyFailures);
> +            if (newFlakyTests.length) {

If there were tooManyFailures, then this would return "false" (or maybe "null") in the future and the .length check would through an error. This should probably become:

    if (newFlakyTests && newFlakyTests.length) { ... }

> Tools/ChangeLog:31
> +        We tell our caller to keep calling until all current failures have been explained and we've
> +        gone through 5 builds without any new flaky tests being identified.

How far back does this normally go? Just 5-10 builds? 100 builds? Is there a limit on how far back this looks or could it go back to the start (if things were flaky).

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.