[webkit-reviews] review granted: [Bug 204090] [EWS] Parse jsc_results.json for JSC tests : [Attachment 383390] Patch after rebasing on ToT

[bugzilla-daemon at webkit.org]

Jonathan Bedard <jbedard at apple.com> has granted Aakash Jain
<aakash_jain at apple.com>'s request for review:
Bug 204090: [EWS] Parse jsc_results.json for JSC tests
https://bugs.webkit.org/show_bug.cgi?id=204090

Attachment 383390: Patch after rebasing on ToT

https://bugs.webkit.org/attachment.cgi?id=383390&action=review




--- Comment #11 from Jonathan Bedard <jbedard at apple.com> ---
Comment on attachment 383390
  --> https://bugs.webkit.org/attachment.cgi?id=383390
Patch after rebasing on ToT

View in context: https://bugs.webkit.org/attachment.cgi?id=383390&action=review

I think blocking this on sorting out the right thing to do with
https://bugs.webkit.org/show_bug.cgi?id=204091 is a bad idea, especially since
we already have infrastructure that relies on the json file you're using here.

I do think that we'll be changing this in the near future, though. Maybe a
FIXME in commandComplete with a bug about standardizing how we think about JSC
test failures is a good idea. I don't think the standardization is in scope of
this problem or https://bugs.webkit.org/show_bug.cgi?id=204091, but it does
need to happen.

> Tools/BuildSlaveSupport/ews-build/steps.py:983
> +	       self.binaryFailures.append('testapi')

I don't feel strongly about this, but we could reduce some duplication here:

for test in ['masm', 'air', 'b3', 'dog', 'api']:
    if jsc_results.get('all{}TestsPassed'.format(test.capitalize())) == False:
	self.binaryFailures.append('test{}'.format(test))