[webkit-reviews] review granted: [Bug 204174] [ews] Add build step to Analyze JSC Tests Results : [Attachment 383502] Patch

[bugzilla-daemon at webkit.org]

Jonathan Bedard <jbedard at apple.com> has granted Aakash Jain
<aakash_jain at apple.com>'s request for review:
Bug 204174: [ews] Add build step to Analyze JSC Tests Results
https://bugs.webkit.org/show_bug.cgi?id=204174

Attachment 383502: Patch

https://bugs.webkit.org/attachment.cgi?id=383502&action=review




--- Comment #6 from Jonathan Bedard <jbedard at apple.com> ---
Comment on attachment 383502
  --> https://bugs.webkit.org/attachment.cgi?id=383502
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=383502&action=review

>> Tools/BuildSlaveSupport/ews-build/steps.py:1081
>> +	    flaky_binary_failures =
first_run_binary_failures.union(second_run_binary_failures) -
first_run_binary_failures.intersection(second_run_binary_failures)
> 
> I'm not convinced that we should give a green bubble with flaky binary
failures....From talking to Zhifei, I get the sense that the binaries don't
have the same sort of flake problems that layout tests or jsc stress tests
have.

Aakash, Zhifei and I talked about this a bit offline.

Binary tests probably aren't flakey, but this is what old EWS did and we don't
actually have concrete data on if test binaries are flakey without
https://bugs.webkit.org/show_bug.cgi?id=204091. We might come back and revisit
this, but for the sake of converting JSC EWS to new EWS, this behavior is the
same as what we were doing before.