<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[215108] trunk/Tools</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/215108">215108</a></dd>
<dt>Author</dt> <dd>aakash_jain@apple.com</dd>
<dt>Date</dt> <dd>2017-04-07 12:06:51 -0700 (Fri, 07 Apr 2017)</dd>
</dl>
<h3>Log Message</h3>
<pre>Add support for test262 JavaScriptCore tests
https://bugs.webkit.org/show_bug.cgi?id=170523
Reviewed by Daniel Bates.
* BuildSlaveSupport/build.webkit.org-config/config.json: Added bot433 and bot434.
* BuildSlaveSupport/build.webkit.org-config/master.cfg:
(RunTest262Tests): Added class to run Test262 tests.
(RunTest262Tests.start): Added.
(RunTest262Tests.countFailures): Method to count the failures.
(Test262Factory): Added Test262 factory class.
* BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:
(RunTest262TestsTest): Added unit tests.
(RunTest262TestsTest.assertResults): Helper method.
(RunTest262TestsTest.test_no_regressions_output): Added test case.
(test_failure_output): Same.
(test_failures_output): Same.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkToolsBuildSlaveSupportbuildwebkitorgconfigconfigjson">trunk/Tools/BuildSlaveSupport/build.webkit.org-config/config.json</a></li>
<li><a href="#trunkToolsBuildSlaveSupportbuildwebkitorgconfigmastercfg">trunk/Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg</a></li>
<li><a href="#trunkToolsBuildSlaveSupportbuildwebkitorgconfigmastercfg_unittestpy">trunk/Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py</a></li>
<li><a href="#trunkToolsChangeLog">trunk/Tools/ChangeLog</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkToolsBuildSlaveSupportbuildwebkitorgconfigconfigjson"></a>
<div class="modfile"><h4>Modified: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/config.json (215107 => 215108)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/config.json 2017-04-07 19:03:59 UTC (rev 215107)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/config.json 2017-04-07 19:06:51 UTC (rev 215108)
</span><span class="lines">@@ -20,6 +20,8 @@
</span><span class="cx"> { "name": "bot417", "platform": "mac-elcapitan" },
</span><span class="cx"> { "name": "bot418", "platform": "mac-elcapitan" },
</span><span class="cx"> { "name": "bot419", "platform": "mac-elcapitan" },
</span><ins>+ { "name": "bot433", "platform": "mac-elcapitan" },
+ { "name": "bot434", "platform": "mac-elcapitan" },
</ins><span class="cx">
</span><span class="cx"> { "name": "bot118", "platform": "mac-sierra" },
</span><span class="cx"> { "name": "bot121", "platform": "mac-sierra" },
</span><span class="lines">@@ -138,7 +140,7 @@
</span><span class="cx"> },
</span><span class="cx"> { "name": "Apple El Capitan Debug (Build)", "type": "Build", "builddir": "elcapitan-debug",
</span><span class="cx"> "platform": "mac-elcapitan", "configuration": "debug", "architectures": ["x86_64"],
</span><del>- "triggers": ["elcapitan-debug-tests-jsc", "elcapitan-debug-tests-wk1", "elcapitan-debug-tests-wk2"],
</del><ins>+ "triggers": ["elcapitan-debug-tests-jsc", "elcapitan-debug-tests-test262", "elcapitan-debug-tests-wk1", "elcapitan-debug-tests-wk2"],
</ins><span class="cx"> "slavenames": ["bot415", "bot416"]
</span><span class="cx"> },
</span><span class="cx"> { "name": "Apple El Capitan CMake Debug (Build)", "type": "Build", "builddir": "elcapitan-cmake-debug",
</span><span class="lines">@@ -160,6 +162,10 @@
</span><span class="cx"> "additionalArguments": ["--no-retry-failures", "--no-sample-on-timeout"],
</span><span class="cx"> "slavenames": ["bot162", "bot165"]
</span><span class="cx"> },
</span><ins>+ { "name": "Apple El Capitan Debug Test262 (Tests)", "type": "Test262", "builddir": "elcapitan-debug-tests-test262",
+ "platform": "mac-elcapitan", "configuration": "debug", "architectures": ["x86_64"],
+ "slavenames": ["bot433"]
+ },
</ins><span class="cx"> { "name": "Apple El Capitan LLINT CLoop (BuildAndTest)", "type": "BuildAndTestLLINTCLoop", "builddir": "elcapitan-cloop-debug",
</span><span class="cx"> "platform": "mac-elcapitan", "configuration": "debug", "architectures": ["x86_64"],
</span><span class="cx"> "slavenames": ["bot169"]
</span><span class="lines">@@ -170,7 +176,7 @@
</span><span class="cx"> },
</span><span class="cx"> { "name": "Apple El Capitan Release (Build)", "type": "Build", "builddir": "elcapitan-release",
</span><span class="cx"> "platform": "mac-elcapitan", "configuration": "release", "architectures": ["x86_64"],
</span><del>- "triggers": ["elcapitan-release-tests-jsc", "elcapitan-release-tests-wk1", "elcapitan-release-tests-wk2", "elcapitan-release-perf"],
</del><ins>+ "triggers": ["elcapitan-release-tests-jsc", "elcapitan-release-tests-test262", "elcapitan-release-tests-wk1", "elcapitan-release-tests-wk2", "elcapitan-release-perf"],
</ins><span class="cx"> "slavenames": ["bot417", "bot418"]
</span><span class="cx"> },
</span><span class="cx"> { "name": "Apple El Capitan Release JSC (Tests)", "type": "TestJSC", "builddir": "elcapitan-release-tests-jsc",
</span><span class="lines">@@ -196,6 +202,10 @@
</span><span class="cx"> "platform": "mac-elcapitan", "configuration": "release", "architectures": ["i386"],
</span><span class="cx"> "slavenames": ["bot419"]
</span><span class="cx"> },
</span><ins>+ { "name": "Apple El Capitan Release Test262 (Tests)", "type": "Test262", "builddir": "elcapitan-release-tests-test262",
+ "platform": "mac-elcapitan", "configuration": "release", "architectures": ["x86_64"],
+ "slavenames": ["bot434"]
+ },
</ins><span class="cx"> {
</span><span class="cx"> "name": "Apple iOS 10 Release (Build)", "type": "Build", "builddir": "ios-10-release",
</span><span class="cx"> "platform": "ios-10", "configuration": "release", "architectures": ["armv7s", "arm64"],
</span><span class="lines">@@ -403,6 +413,12 @@
</span><span class="cx"> { "type": "Triggerable", "name": "elcapitan-release-perf",
</span><span class="cx"> "builderNames": ["Apple El Capitan Release WK2 (Perf)"]
</span><span class="cx"> },
</span><ins>+ { "type": "Triggerable", "name": "elcapitan-debug-tests-test262",
+ "builderNames": ["Apple El Capitan Debug Test262 (Tests)"]
+ },
+ { "type": "Triggerable", "name": "elcapitan-release-tests-test262",
+ "builderNames": ["Apple El Capitan Release Test262 (Tests)"]
+ },
</ins><span class="cx"> { "type": "Triggerable", "name": "ios-simulator-10-release-tests-wk1",
</span><span class="cx"> "builderNames": ["Apple iOS 10 Simulator Release WK1 (Tests)"]
</span><span class="cx"> },
</span></span></pre></div>
<a id="trunkToolsBuildSlaveSupportbuildwebkitorgconfigmastercfg"></a>
<div class="modfile"><h4>Modified: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg (215107 => 215108)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg 2017-04-07 19:03:59 UTC (rev 215107)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg 2017-04-07 19:06:51 UTC (rev 215108)
</span><span class="lines">@@ -322,6 +322,25 @@
</span><span class="cx"> return RunJavaScriptCoreTests.start(self)
</span><span class="cx">
</span><span class="cx">
</span><ins>+class RunTest262Tests(TestWithFailureCount):
+ name = "test262-test"
+ description = ["test262-tests running"]
+ descriptionDone = ["test262-tests"]
+ failedTestsFormatString = "%d Test262 test%s failed"
+ command = ["perl", "./Tools/Scripts/run-jsc-stress-tests", WithProperties("--%(configuration)s"), "JSTests/test262.yaml"]
+
+ def start(self):
+ appendCustomBuildFlags(self, self.getProperty('platform'), self.getProperty('fullPlatform'))
+ return shell.Test.start(self)
+
+ def countFailures(self, cmd):
+ logText = cmd.logs['stdio'].getText()
+ matches = re.findall(r'\(failed (\d+)\)', logText)
+ if matches:
+ return int(matches[-1])
+ return 0
+
+
</ins><span class="cx"> class RunWebKitTests(shell.Test):
</span><span class="cx"> name = "layout-test"
</span><span class="cx"> description = ["layout-tests running"]
</span><span class="lines">@@ -893,6 +912,13 @@
</span><span class="cx"> self.addStep(ExtractBuiltProduct())
</span><span class="cx"> self.addStep(RunJavaScriptCoreTests())
</span><span class="cx">
</span><ins>+class Test262Factory(Factory):
+ def __init__(self, platform, configuration, architectures, additionalArguments=None, SVNMirror=None):
+ Factory.__init__(self, platform, configuration, architectures, False, additionalArguments, SVNMirror)
+ self.addStep(DownloadBuiltProduct())
+ self.addStep(ExtractBuiltProduct())
+ self.addStep(RunTest262Tests())
+
</ins><span class="cx"> class TestWebKit1Factory(TestFactory):
</span><span class="cx"> LayoutTestClass = RunWebKit1Tests
</span><span class="cx">
</span></span></pre></div>
<a id="trunkToolsBuildSlaveSupportbuildwebkitorgconfigmastercfg_unittestpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py (215107 => 215108)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py 2017-04-07 19:03:59 UTC (rev 215107)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py 2017-04-07 19:06:51 UTC (rev 215108)
</span><span class="lines">@@ -140,6 +140,34 @@
</span><span class="cx"> 5 failures found.""")
</span><span class="cx">
</span><span class="cx">
</span><ins>+class RunTest262TestsTest(unittest.TestCase):
+ def assertResults(self, expected_result, expected_text, rc, stdio):
+ cmd = StubRemoteCommand(rc, stdio)
+ step = RunTest262Tests()
+ step.commandComplete(cmd)
+ actual_results = step.evaluateCommand(cmd)
+ actual_text = step.getText2(cmd, actual_results)
+
+ self.assertEqual(expected_result, actual_results)
+ self.assertEqual(actual_text, expected_text)
+
+ def test_no_regressions_output(self):
+ self.assertResults(SUCCESS, ["test262-test"], 0, """Using the following jsc path: /WebKitBuild/Release/jsc
+168/168 """)
+
+ def test_failure_output(self):
+ self.assertResults(FAILURE, ["1 Test262 test failed"], 1, """Using the following jsc path: /WebKitBuild/Release/jsc
+test262.yaml/test262/test/built-ins/Array/from/iter-set-elem-prop-err.js.default: ERROR: Unexpected exit code: 0
+test262.yaml/test262/test/built-ins/Array/from/iter-set-elem-prop-err.js.default-strict: ERROR: Unexpected exit code: 0
+...
+43768/43768 (failed 1) """)
+
+ def test_failures_output(self):
+ self.assertResults(FAILURE, ["75 Test262 tests failed"], 75, """Using the following jsc path: /WebKitBuild/Release/jsc
+...
+43768/43768 (failed 75) """)
+
+
</ins><span class="cx"> class RunLLINTCLoopTestsTest(unittest.TestCase):
</span><span class="cx"> def assertResults(self, expected_result, expected_text, rc, stdio):
</span><span class="cx"> cmd = StubRemoteCommand(rc, stdio)
</span><span class="lines">@@ -374,6 +402,7 @@
</span><span class="cx"> 'Apple El Capitan CMake Debug (Build)' :['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile-webkit'],
</span><span class="cx"> 'Apple El Capitan Debug (Build)' : ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile-webkit', 'archive-built-product', 'upload', 'trigger'],
</span><span class="cx"> 'Apple El Capitan Debug JSC (Tests)' : ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'jscore-test'],
</span><ins>+ 'Apple El Capitan Debug Test262 (Tests)' : ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'test262-test'],
</ins><span class="cx"> 'Apple El Capitan Debug WK1 (Tests)' : ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
</span><span class="cx"> 'Apple El Capitan Debug WK2 (Tests)' : ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
</span><span class="cx"> 'Apple El Capitan LLINT CLoop (BuildAndTest)' : ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile-webkit', 'webkit-jsc-cloop-test'],
</span><span class="lines">@@ -381,6 +410,7 @@
</span><span class="cx"> 'Apple El Capitan Release (Build)' : ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile-webkit', 'archive-built-product', 'upload', 'trigger'],
</span><span class="cx"> 'Apple El Capitan Release WK2 (Perf)' : ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'perf-test'],
</span><span class="cx"> 'Apple El Capitan Release JSC (Tests)' : ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'jscore-test'],
</span><ins>+ 'Apple El Capitan Release Test262 (Tests)' : ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'test262-test'],
</ins><span class="cx"> 'Apple El Capitan Release WK1 (Tests)' : ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
</span><span class="cx"> 'Apple El Capitan Release WK2 (Tests)' : ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
</span><span class="cx">
</span></span></pre></div>
<a id="trunkToolsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Tools/ChangeLog (215107 => 215108)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/ChangeLog 2017-04-07 19:03:59 UTC (rev 215107)
+++ trunk/Tools/ChangeLog 2017-04-07 19:06:51 UTC (rev 215108)
</span><span class="lines">@@ -1,3 +1,23 @@
</span><ins>+2017-04-07 Aakash Jain <aakash_jain@apple.com>
+
+ Add support for test262 JavaScriptCore tests
+ https://bugs.webkit.org/show_bug.cgi?id=170523
+
+ Reviewed by Daniel Bates.
+
+ * BuildSlaveSupport/build.webkit.org-config/config.json: Added bot433 and bot434.
+ * BuildSlaveSupport/build.webkit.org-config/master.cfg:
+ (RunTest262Tests): Added class to run Test262 tests.
+ (RunTest262Tests.start): Added.
+ (RunTest262Tests.countFailures): Method to count the failures.
+ (Test262Factory): Added Test262 factory class.
+ * BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:
+ (RunTest262TestsTest): Added unit tests.
+ (RunTest262TestsTest.assertResults): Helper method.
+ (RunTest262TestsTest.test_no_regressions_output): Added test case.
+ (test_failure_output): Same.
+ (test_failures_output): Same.
+
</ins><span class="cx"> 2017-04-07 Caio Lima <ticaiolima@gmail.com>
</span><span class="cx">
</span><span class="cx"> [JSC] Change --debug option to --debugger
</span></span></pre>
</div>
</div>
</body>
</html>