[Webkit-unassigned] [Bug 218618] New: [ews] Patch which break layout-test shouldn't cause infinite RETRY loop

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=218618

            Bug ID: 218618
           Summary: [ews] Patch which break layout-test shouldn't cause
                    infinite RETRY loop
           Product: WebKit
           Version: Other
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: aakash_jain at apple.com

Currently we retry layout-tests when there is an exception in running the tests (exit code 254). We do it to handle any unexpected infrastructure issues (like simulators not booting up properly). We keep retrying until the issue is fixed.

However, because of this infinite retry, patches which actually break layout-test are also retried, and go into an infinite retry loop (until the patch is obsolete, r-, bug is closed, or someone manually stops the build). e.g.: https://bugs.webkit.org/show_bug.cgi?id=218352#c2 caused continuos retries (https://ews-build.webkit.org/#/builders/30/builds/21779, https://ews-build.webkit.org/#/builders/30/builds/21778, https://ews-build.webkit.org/#/builders/30/builds/21777 etc.)

We shouldn't automatically retry immediately when layout-tests unexpectedly fail. We should re-run the tests on clean tree as well. If the clean tree passes, it indicates that the issue is with the patch, and the build should be marked as FAILURE (and red ews bubble). Otherwise (if clean-tree results are also failure) we should retry the build.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20201105/ba7ecc7e/attachment-0001.htm>