[webkit-dev] Recent EWS improvements

[Aakash Jain]

Hi Everyone,

I just wanted to update everyone with the recent improvements I have made to new EWS. As always, please feel encouraged to provide any feedback (either by filing bugs or contacting me directly).

New Features:
EWS status-bubble now display position in queue while patch is waiting to be processed
Added webkitpy and bindings-tests EWS (moved from old to new EWS)
Status bubbles for webkitpy and bindings-tests EWS now display the exact test failures in hover-over message (https://webkit.org/b/197395 <https://webkit.org/b/197395>, https://webkit.org/b/197423 <https://webkit.org/b/197423>)
Added support for 'new EWS' in webkit-patch tool
Added 'EWS Build Archives' (similar to 'WebKit Build Archives' https://webkit.org/blog/7978/introducing-webkit-build-archives/ <https://webkit.org/blog/7978/introducing-webkit-build-archives/>). For every patch uploaded to Bugzilla, EWS builders build the patch for various platforms (currently macOS and iOS) and upload the archives to S3. These archives are available to download by anyone (for 14 days). The S3 URL is in corresponding build (e.g.: notice 'uploaded archive' link in https://ews-build.webkit.org/#/builders/7/builds/2477 <https://ews-build.webkit.org/#/builders/7/builds/2477>). So, if for any reason, you want to get a built archive for your patch, you can simply upload the patch to Bugzilla. (Note that if there is interest in this, we can enhance it further)

Infrastructure Improvements:
Flakiness in API tests has been reduced (thanks to many WebKit developers)
Infrastructure improvements to prevent build failure due to "worker not pinged" (e.g.: https://ews-build.webkit.org/#/builders/9/builds/332 <https://ews-build.webkit.org/#/builders/9/builds/332>)
New EWS polls bugzilla more frequently https://webkit.org/b/197138 <https://webkit.org/b/197138>
Configured DEBUG mode appropriately for Production and Development env https://webkit.org/b/197700 <https://webkit.org/b/197700>
Ensured that Buildbot worker logs are not lost on restarting worker
Do not run clean build by default on EWS builders (to improve efficiency) https://webkit.org/b/196897 <https://webkit.org/b/196897>
build.webkit.org <http://build.webkit.org/> and ews-build.webkit.org <http://ews-build.webkit.org/> starting sharing code (although very little as of now, however the plan is to share more code)
Added migrations file to repository https://webkit.org/b/197729 <https://webkit.org/b/197729>
Added EWS bots information to Internal scripts to easily monitor bots
Added more unit-tests

Bug fixes:
Clicking 'submit to new ews' doesn't reload status-bubble https://webkit.org/b/196675 <https://webkit.org/b/196675>
Clicking on white bubble navigates to page with only bubbles https://webkit.org/b/197520 <https://webkit.org/b/197520>
Submit to EWS buttons are not aligned properly with status-bubbles https://webkit.org/b/197139 <https://webkit.org/b/197139>
Status bubble should turn orange when any build step fails https://webkit.org/b/197812 <https://webkit.org/b/197812>
Handle bug titles with unicode characters https://webkit.org/b/196802 <https://webkit.org/b/196802>
Scripts using Buildbot API have CORS error https://webkit.org/b/196709 <https://webkit.org/b/196709>
PrintConfiguration should display Xcode version instead of SDKVersion https://webkit.org/b/196780 <https://webkit.org/b/196780>
Trigger queues only after uploading the archive https://webkit.org/b/197180 <https://webkit.org/b/197180>
Do not upload archive when Compile Fails https://webkit.org/b/196674 <https://webkit.org/b/196674>
Exception while loading status-bubble when no build step has started https://webkit.org/b/196676 <https://webkit.org/b/196676>
Use singular verb in failure description in case of single api test failure https://webkit.org/b/197013 <https://webkit.org/b/197013>
EWS should clearly indicate flaky test failures https://webkit.org/b/196947 <https://webkit.org/b/196947>
Use explicit imports instead of wildcard imports https://webkit.org/b/197194 <https://webkit.org/b/197194>
New EWS: patches on recently added queues listed as #1 for older bugs https://webkit.org/b/197496 <https://webkit.org/b/197496>
Improved summary text for various build steps

Interesting info: Since last month, 'EWS for API tests' prevented API test breakage on 50+ patches (https://ews-build.webkit.org/api/v2/builders/3/builds?state_string__contains=new%20API%20Test&property=bug_id&order=-number <https://ews-build.webkit.org/api/v2/builders/3/builds?state_string__contains=new%20API%20Test&property=bug_id&order=-number>).

Thanks
Aakash

> On Apr 4, 2019, at 10:00 PM, Aakash Jain <aakash_jain at apple.com> wrote:
> 
> Introducing brand new EWS
> 
> with EWS for API Tests (for macOS and iOS)
> 
> and EWS for WebKitPerl Tests!
> 
> 
> Starting today, when you upload a patch to bugs.webkit.org <http://bugs.webkit.org/>, you will see few more bubbles (for API tests and webkitperl tests). You might also see additional button 'Submit to new EWS' (if the patch doesn't have r? flag).
> 
> The new EWS comes with many new features and there are lot more I want to add. But, I don't want you guys to wait more to start getting the benefits. That's why I am rolling it out in phases, starting with EWS for API Tests and WebKitPerl Tests. These are tests which are not currently covered by the existing EWS. Next step would be to move queues from existing EWS to new EWS one by one, with the eventual goal of moving over everything to new EWS.
> 
> 
> Why new EWS?
> The existing EWS has certain architectural limitations. One of the prominent limitation is that there is no concept of building and testing the patch on different queues. If we have three queues: WK1 tests, WK2 tests and API tests, all three queues would need to compile WebKit independently. So WebKit would be compiled thrice instead of once. This is inefficient and thereby require more hardware.
> 
> The new EWS has separate builder and tester queues. Builder queues build once and upload the archive. Multiple tester queues download that same archive and run tests on that. That way WebKit is compiled only once, and re-used on multiple tester queues. This improves system efficiency and allows us to add new test queues with substantially less hardware.
> 
> The new EWS uses Buildbot at the back-end, which is a production-level CI system. It is easier to maintain and automatically provide various features like historical build logs, real-time log streaming, easier bot management, ability to retry a build etc. Plus, it’s a system most of you are already familiar with (build.webkit.org <http://build.webkit.org/>).
> 
> 
> How can you contribute:
> If you are interested in contributing, the source code is located at:
> ews-build (Buildbot): Tools/BuildSlaveSupport/ews-build
> ews-app (web-app): Tools/BuildSlaveSupport/ews-app
> 
> Detailed instructions are at: https://trac.webkit.org/wiki/EarlyWarningSystem#ContributingtoEarlyWarningSystem <https://trac.webkit.org/wiki/EarlyWarningSystem#ContributingtoEarlyWarningSystem>
> 
> 
> Upcoming features:
> - status-bubble should display position in queue https://bugs.webkit.org/show_bug.cgi?id=196607 <https://bugs.webkit.org/show_bug.cgi?id=196607>
> - EWS should comment on Bugzilla bugs about failures https://bugs.webkit.org/show_bug.cgi?id=196598 <https://bugs.webkit.org/show_bug.cgi?id=196598>
> - EWS should have a way to retry a patch https://bugs.webkit.org/show_bug.cgi?id=196599 <https://bugs.webkit.org/show_bug.cgi?id=196599>
> - Security EWS https://bugs.webkit.org/show_bug.cgi?id=196605 <https://bugs.webkit.org/show_bug.cgi?id=196605>
> 
> 
> If you notice any issue, please feel free to file bugs (and assign to me).
> 
> Thanks & Regards
> Aakash Jain

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20190522/9d40a942/attachment.html>