[Webkit-unassigned] [Bug 201607] New: [EWS] API testers can go in an infinite RETRY loop when there is build failure in trunk

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=201607

            Bug ID: 201607
           Summary: [EWS] API testers can go in an infinite RETRY loop
                    when there is build failure in trunk
           Product: WebKit
           Version: Other
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: aakash_jain at apple.com

If there is a build failure on top of tree, and the patch being tested have API test failures, then the API tester would fail to build and RETRY the build, and keep RETRYing, until the patch is obsoleted/closed.

This happens in very specific scenario, when following conditions are met:
1) builder was able to build irrespective of ToT having build failure (because it does incremental build, not clean build by default).
2) API-tester found a test failure with the patch.
3) API-tester failed to build top-of-tree (since it does clean-build).

In that case, tester will retry the build and since it's testing the exact same revision, it will fail again and keep RETRYing in this manner. We shouldn't RETRY in such cases.

For e.g.: 
For Patch 378021 (in https://bugs.webkit.org/show_bug.cgi?id=201471), Builder was able to build in  https://ews-build.webkit.org/#/builders/13/builds/8775, In that build the top-of-tree was at 1d2cc7a93bcdec77250d1c4ab9686e5cc32c170a (equivalent to r249500), which had a build failure at that time (Build was broken in r249498 and fixed in r249525 in https://bugs.webkit.org/show_bug.cgi?id=201468). Builder was able to build since it does incremental build, and the patch probably didn't modify any files in the part which was broken.

The corresponding API tester noticed API test failures, and tried to build ToT and failed and keep retrying, 
e.g.: 
https://ews-build.webkit.org/#/builders/9/builds/7585
https://ews-build.webkit.org/#/builders/9/builds/7589
https://ews-build.webkit.org/#/builders/9/builds/7594
https://ews-build.webkit.org/#/builders/9/builds/7599
https://ews-build.webkit.org/#/builders/9/builds/7603
https://ews-build.webkit.org/#/builders/9/builds/8094

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20190909/72985d5c/attachment.html>