[Webkit-unassigned] [Bug 137892] Patch is spinning EWS for no obvious reason

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=137892

Jake Nielsen <jake.nielsen.webkit at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|webkit-unassigned at lists.web |jake.nielsen.webkit at gmail.c
                   |kit.org                     |om

--- Comment #1 from Jake Nielsen <jake.nielsen.webkit at gmail.com> ---
Looks like this is the culprit:

(patchanalysistask.py:208)

if (not first_results.did_exceed_test_failure_limit() and
    not second_results.did_exceed_test_failure_limit() and
    self._results_failed_different_tests(first_results, second_results)):
    # We could report flaky tests here, but we would need to be careful
    # to use similar checks to ExpectedFailures._can_trust_results
    # to make sure we don't report constant failures as flakes when
    # we happen to hit the --exit-after-N-failures limit.
    # See https://bugs.webkit.org/show_bug.cgi?id=51272
    return False

Neither exceeds the failure limit, and they do fail different tests, so it defers. It should probably report the flakiness that it can, and fall through.

In general I do think that's just a band-aid though, this whole function needs some stream-lining. It's difficult to reason about because it's got so many special cases that get filtered out one-by-one. I think we can do better.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20141020/0cfe5483/attachment-0002.html>