[Webkit-unassigned] [Bug 230006] New: [build.webkit.org] [ews-build.webkit.org] Only try to download from S3 on the production server

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=230006

            Bug ID: 230006
           Summary: [build.webkit.org] [ews-build.webkit.org] Only try to
                    download from S3 on the production server
           Product: WebKit
           Version: WebKit Nightly Build
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: clopez at igalia.com
                CC: aakash_jain at apple.com, angelos at igalia.com,
                    dpino at igalia.com

Uploading the built product to S3 is only possible on the official deployment (since it has the keys, etc)
So on tests deployments we need to find a different way of passing the built product between workers.

r269261 implemented this for ews-build.webkit.org and then r269634 implemented it for build.webkit.org

After this patches on a test deployment the product is not uploaded to S3, is only uploaded from the worker to the main buildbot server (master)

However, the worker still tries to download the test product from S3 and only if fails it fall-backs to download it from the master.

This is an issue, because the URL identifiers on S3 are not random, so it can happen that the S3 download on a test deployment doesn't fail but instead downloads the built product from the official deployment

For build.webkit.org the only variable in the URL is the bot name, architecture and svn revision and for ews-build.webkit.org is the same but instead of the svn revision the patch number.

And why is a problem downloading the built product from the official deployment instead of the test one?

There are several problems with this. For example there is no guarantee that on a test deployment the built configurations are the same than on the official one.
So I think is safe to assume that the user would want to consume the built products from her own test deployment.

Also, even when testing the same build configurations, this can also causes issue on testers, see this example:

1. Official GTK build EWS builds patch 437292 on top of 91691c326150 (r282009) <https://ews-build.webkit.org/#/builders/36/builds/43955>
2. Uploads product it to https://s3-us-west-2.amazonaws.com/ews-archives.webkit.org/gtk-x86_64-release/437292.zip
3. Triggers GTK API test EWS
4. Official GTK API test EWS downloads patch 437292 for testing and correctly checkouts 91691c326150 (r282009) <https://ews-build.webkit.org/#/builders/34/builds/42792>

A. Unofficial GTK build EWS builds patch 437292 on top of 1ee2b9feb0a3 (r282020) <https://ews-build.webkit-uat.org/#/builders/35/builds/107607>
B. It don't uploads the product (transfer-to-s3 step is skipped)
C. Triggers GTK WK2 Layout tests EWS
D. Unofficial GTK WK2 Layout test EWS downloads patch 437292 for testing, but it downloads the patch from the official buildbot instead of the one from UAT (See 1.). That patch was built on top of r282009. However, this buildbot checkouts 1ee2b9feb0a3 (r282020) (which was the checkout used by the UAT buildbot GTK builder)
E. So we end with this GTK WK2 Layout tests worker checked out at r282009 testing a patch that was built on top of r282020 <https://ews-build.webkit-uat.org/#/builders/34/builds/41403>

What is the problem with this?

- r282020 removed several failures from the TestExpectations file because it fixed those tests
- But since the built-product we are testing was built on top of r282009 it still don't has the fixes implemented by r282020
- However, on our checkout the failures are removed from the TestExpectations file (because we are checking r282020 instead of r282009).
- This causes unexpected failures on this Unofficial GTK WK2 Layout test EWS run <https://ews-build.webkit-uat.org/#/builders/34/builds/41403>, the tests imported/w3c/web-platform-tests/css/css-color-adjust/rendering/dark-color-scheme/color-scheme-iframe-background-mismatch-opaque-cross-origin.sub.html and imported/w3c/web-platform-tests/css/css-color-adjust/rendering/dark-color-scheme/color-scheme-iframe-background-mismatch-opaque.html are failing because the built-product tested doesn't match the checked-out base version.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20210907/b1b88684/attachment.htm>