[Webkit-unassigned] [Bug 51371] New: commit-queue can't upload failure diffs from the first flake on a double-flake

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=51371

           Summary: commit-queue can't upload failure diffs from the first
                    flake on a double-flake
           Product: WebKit
           Version: 528+ (Nightly build)
          Platform: PC
        OS/Version: Mac OS X 10.5
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
        AssignedTo: webkit-unassigned at lists.webkit.org
        ReportedBy: eric at webkit.org
                CC: abarth at webkit.org


commit-queue can't upload failure diffs from the first flake on a double-flake

def _test_patch(self):
    if self._patch.is_rollout():
        return True
    if self._test():
        return True

    first_failing_tests = self._failing_tests_from_last_run()
    if self._test():
        self._report_flaky_tests(first_failing_tests)
        return True

    second_failing_tests = self._failing_tests_from_last_run()
    if first_failing_tests != second_failing_tests:
        self._report_flaky_tests(first_failing_tests + second_failing_tests)
        return False

    if self._build_and_test_without_patch():
        raise self._script_error  # The error from the previous ._test() run is real, report it.
    return False  # Tree must be red, just retry later.


Currently _failing_tests_from_last_run() only returns the test names (paths) for the failures.  It does not bottle up the results information.  So when we run the tests a second time, old-run-webkit-tests will delete the results from the first run.  Then when we go to try and report the test as flaky, we don't have any results diff to upload.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.