<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[160016] trunk/Tools</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/160016">160016</a></dd>
<dt>Author</dt> <dd>mhahnenberg@apple.com</dd>
<dt>Date</dt> <dd>2013-12-03 10:51:12 -0800 (Tue, 03 Dec 2013)</dd>
</dl>
<h3>Log Message</h3>
<pre>run-jsc-stress-tests only supports host environments that have make installed
https://bugs.webkit.org/show_bug.cgi?id=124550
Reviewed by Darin Adler.
This might not be the case for all hosts, so this patch implements an alternate "backend"
for run-jsc-stress-tests to use normal shell commands rather than Makefiles. To remain at
least somewhat competitive with the make-based test runner, the shell backend uses subshells
run in the background to allow tests to run in parallel. Since the concurrency primitives
in shell scripting are rather coarse, the overhead of this parallelism is higher than that
of the make-based runner.
* Scripts/jsc-stress-test-helpers/shell-runner.sh: Added. This is the runner that is copied into
the bundle and controls all of the parallel aspects of the shell-based test runner.
* Scripts/run-jsc-stress-tests:</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkToolsChangeLog">trunk/Tools/ChangeLog</a></li>
<li><a href="#trunkToolsScriptsrunjscstresstests">trunk/Tools/Scripts/run-jsc-stress-tests</a></li>
</ul>
<h3>Added Paths</h3>
<ul>
<li><a href="#trunkToolsScriptsjscstresstesthelpersshellrunnersh">trunk/Tools/Scripts/jsc-stress-test-helpers/shell-runner.sh</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkToolsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Tools/ChangeLog (160015 => 160016)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/ChangeLog 2013-12-03 18:44:32 UTC (rev 160015)
+++ trunk/Tools/ChangeLog 2013-12-03 18:51:12 UTC (rev 160016)
</span><span class="lines">@@ -1,3 +1,21 @@
</span><ins>+2013-12-03 Mark Hahnenberg <mhahnenberg@apple.com>
+
+ run-jsc-stress-tests only supports host environments that have make installed
+ https://bugs.webkit.org/show_bug.cgi?id=124550
+
+ Reviewed by Darin Adler.
+
+ This might not be the case for all hosts, so this patch implements an alternate "backend"
+ for run-jsc-stress-tests to use normal shell commands rather than Makefiles. To remain at
+ least somewhat competitive with the make-based test runner, the shell backend uses subshells
+ run in the background to allow tests to run in parallel. Since the concurrency primitives
+ in shell scripting are rather coarse, the overhead of this parallelism is higher than that
+ of the make-based runner.
+
+ * Scripts/jsc-stress-test-helpers/shell-runner.sh: Added. This is the runner that is copied into
+ the bundle and controls all of the parallel aspects of the shell-based test runner.
+ * Scripts/run-jsc-stress-tests:
+
</ins><span class="cx"> 2013-12-03 Tamas Gergely <tgergely.u-szeged@partner.samsung.com>
</span><span class="cx">
</span><span class="cx"> Remove function from TextChecker
</span></span></pre></div>
<a id="trunkToolsScriptsjscstresstesthelpersshellrunnersh"></a>
<div class="addfile"><h4>Added: trunk/Tools/Scripts/jsc-stress-test-helpers/shell-runner.sh (0 => 160016)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/jsc-stress-test-helpers/shell-runner.sh (rev 0)
+++ trunk/Tools/Scripts/jsc-stress-test-helpers/shell-runner.sh 2013-12-03 18:51:12 UTC (rev 160016)
</span><span class="lines">@@ -0,0 +1,76 @@
</span><ins>+# Copyright (C) 2013 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE AND ITS CONTRIBUTORS "AS IS" AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL APPLE OR ITS CONTRIBUTORS BE LIABLE FOR ANY
+# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+numProcs=`sysctl -n hw.activecpu`
+if [ $? -gt 0 ]
+then
+ numProcs=`nproc --all 2>/dev/null`
+ if [ $? -gt 0 ]
+ then
+ numProcs=1
+ fi
+fi
+
+indexFile=".index"
+testList=".all_tests.txt"
+tempFile=".temp.txt"
+lockDir=".lock_dir"
+
+trap "kill -9 0" SIGINT
+
+echo 0 > ${indexFile}
+ls test_script_* > ${testList}
+
+function lock_test_list() {
+ until mkdir ${lockDir} 2> /dev/null; do sleep 0; done
+}
+
+function unlock_test_list() {
+ rmdir ${lockDir}
+}
+
+total=`wc -l < "${testList}" | sed 's/ //g'`
+for proc in `seq ${numProcs}`
+do
+ (
+ lock_test_list
+ while [ -s ${testList} ]
+ do
+ index=`cat ${indexFile}`
+ index=$((index + 1))
+ echo "${index}" > ${indexFile}
+ printf "\r ${index}/${total}"
+
+ nextTest=`tail -n 1 ${testList}`
+ sed '$d' < ${testList} > ${tempFile}
+ mv ${tempFile} ${testList}
+ unlock_test_list
+
+ sh ${nextTest} > /dev/null
+
+ lock_test_list
+ done
+ unlock_test_list
+ )&
+done
+wait
</ins></span></pre></div>
<a id="trunkToolsScriptsrunjscstresstests"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/run-jsc-stress-tests (160015 => 160016)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/run-jsc-stress-tests 2013-12-03 18:44:32 UTC (rev 160015)
+++ trunk/Tools/Scripts/run-jsc-stress-tests 2013-12-03 18:51:12 UTC (rev 160016)
</span><span class="lines">@@ -68,7 +68,7 @@
</span><span class="cx"> end
</span><span class="cx">
</span><span class="cx"> def mysys(*cmd)
</span><del>- printCommandArray(*cmd)
</del><ins>+ printCommandArray(*cmd) if $verbosity >= 1
</ins><span class="cx"> raise "Command failed: #{$?.inspect}" unless system(*cmd)
</span><span class="cx"> end
</span><span class="cx">
</span><span class="lines">@@ -92,6 +92,7 @@
</span><span class="cx"> $bundle = nil
</span><span class="cx"> $tarball = false
</span><span class="cx"> $copyVM = false
</span><ins>+$testRunnerType = :make
</ins><span class="cx">
</span><span class="cx"> def usage
</span><span class="cx"> puts "run-jsc-stress-tests -j <shell path> <collections path> [<collections path> ...]"
</span><span class="lines">@@ -102,6 +103,8 @@
</span><span class="cx"> puts "--verbose (-v) Print more things while running."
</span><span class="cx"> puts "--run-bundle Runs a bundle previously created by run-jsc-stress-tests."
</span><span class="cx"> puts "--tarball Creates a tarball of the final bundle."
</span><ins>+ puts "--shell-runner Uses the shell-based test runner instead of the default make-based runner."
+ puts " In general the shell runner is slower than the make runner."
</ins><span class="cx"> puts "--help (-h) Print this message."
</span><span class="cx"> exit 1
</span><span class="cx"> end
</span><span class="lines">@@ -113,6 +116,7 @@
</span><span class="cx"> ['--run-bundle', GetoptLong::REQUIRED_ARGUMENT],
</span><span class="cx"> ['--tarball', GetoptLong::NO_ARGUMENT],
</span><span class="cx"> ['--force-vm-copy', GetoptLong::NO_ARGUMENT],
</span><ins>+ ['--shell-runner', GetoptLong::NO_ARGUMENT],
</ins><span class="cx"> ['--verbose', '-v', GetoptLong::NO_ARGUMENT]).each {
</span><span class="cx"> | opt, arg |
</span><span class="cx"> case opt
</span><span class="lines">@@ -133,6 +137,8 @@
</span><span class="cx"> $copyVM = true
</span><span class="cx"> when '--force-vm-copy'
</span><span class="cx"> $copyVM = true
</span><ins>+ when '--shell-runner'
+ $testRunnerType = :shell
</ins><span class="cx"> end
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -381,7 +387,7 @@
</span><span class="cx"> end
</span><span class="cx">
</span><span class="cx"> def reproScriptCommand
</span><del>- # We have to find our way back to the .parallel directory since that's where all of the relative
</del><ins>+ # We have to find our way back to the .runner directory since that's where all of the relative
</ins><span class="cx"> # paths assume they start out from.
</span><span class="cx"> script = "CURRENT_DIR=\"$( cd \"$( dirname \"${BASH_SOURCE[0]}\" )\" && pwd )\"\n"
</span><span class="cx"> script += "cd $CURRENT_DIR\n"
</span><span class="lines">@@ -389,7 +395,7 @@
</span><span class="cx"> | pathComponent |
</span><span class="cx"> script += "cd ..\n"
</span><span class="cx"> }
</span><del>- script += "cd .parallel\n"
</del><ins>+ script += "cd .runner\n"
</ins><span class="cx">
</span><span class="cx"> script += "export DYLD_FRAMEWORK_PATH=$(cd ../#{$frameworkPath.dirname}; pwd)\n"
</span><span class="cx"> IMPORTANT_ENVS.each {
</span><span class="lines">@@ -916,13 +922,54 @@
</span><span class="cx"> end
</span><span class="cx"> end
</span><span class="cx">
</span><del>-def prepareParallelTestRunner
</del><ins>+def prepareTestRunner
+ raise if $bundle
+
+ $runlist.each_with_index {
+ | plan, index |
+ plan.index = index
+ }
+
+ Dir.mkdir($runnerDir) unless $runnerDir.directory?
+ toDelete = []
+ Dir.foreach($runnerDir) {
+ | filename |
+ if filename =~ /^test_/
+ toDelete << filename
+ end
+ }
+
+ toDelete.each {
+ | filename |
+ File.unlink($runnerDir + filename)
+ }
+
+ $runlist.each {
+ | plan |
+ plan.writeRunScript($runnerDir + "test_script_#{plan.index}")
+ }
+
+ case $testRunnerType
+ when :make
+ prepareMakeTestRunner
+ when :shell
+ prepareShellTestRunner
+ else
+ raise "Unknown test runner type: #{$testRunnerType.to_s}"
+ end
+end
+
+def prepareShellTestRunner
+ FileUtils.cp SCRIPTS_PATH + "jsc-stress-test-helpers" + "shell-runner.sh", $runnerDir + "runscript"
+end
+
+def prepareMakeTestRunner
</ins><span class="cx"> # The goals of our parallel test runner are scalability and simplicity. The
</span><span class="cx"> # simplicity part is particularly important. We don't want to have to have
</span><span class="cx"> # a full-time contributor just philosophising about parallel testing.
</span><span class="cx"> #
</span><span class="cx"> # As such, we just pass off all of the hard work to 'make'. This creates a
</span><del>- # dummy directory ("$outputDir/.parallel") in which we create a dummy
</del><ins>+ # dummy directory ("$outputDir/.runner") in which we create a dummy
</ins><span class="cx"> # Makefile. The Makefile has an 'all' rule that depends on all of the tests.
</span><span class="cx"> # That is, for each test we know we will run, there is a rule in the
</span><span class="cx"> # Makefile and 'all' depends on it. Running 'make -j <whatever>' on this
</span><span class="lines">@@ -945,40 +992,19 @@
</span><span class="cx"> # continue past that failure and complete all the tests anyway.
</span><span class="cx"> #
</span><span class="cx"> # In the end, this script collects all of the failures by searching for
</span><del>- # files in the .parallel directory whose name matches /^test_fail_/, where
</del><ins>+ # files in the .runner directory whose name matches /^test_fail_/, where
</ins><span class="cx"> # the thing after the 'fail_' is the test index. Those are the files that
</span><span class="cx"> # would be created by the test scripts if they detect failure. We're
</span><span class="cx"> # basically using the filesystem as a concurrent database of test failures.
</span><span class="cx"> # Even if two tests fail at the same time, since they're touching different
</span><span class="cx"> # files we won't miss any failures.
</span><del>-
</del><span class="cx"> runIndices = []
</span><del>- $runlist.each_with_index {
- | plan, index |
- runIndices << index
- plan.index = index
- }
-
- Dir.mkdir($parallelDir) unless $parallelDir.directory?
- toDelete = []
- Dir.foreach($parallelDir) {
- | filename |
- if filename =~ /^test_/
- toDelete << filename
- end
- }
-
- toDelete.each {
- | filename |
- File.unlink($parallelDir + filename)
- }
-
</del><span class="cx"> $runlist.each {
</span><span class="cx"> | plan |
</span><del>- plan.writeRunScript($parallelDir + "test_script_#{plan.index}")
</del><ins>+ runIndices << plan.index
</ins><span class="cx"> }
</span><span class="cx">
</span><del>- File.open($parallelDir + "Makefile", "w") {
</del><ins>+ File.open($runnerDir + "Makefile", "w") {
</ins><span class="cx"> | outp |
</span><span class="cx"> outp.puts("all: " + runIndices.map{|v| "test_done_#{v}"}.join(' '))
</span><span class="cx"> runIndices.each {
</span><span class="lines">@@ -1004,17 +1030,34 @@
</span><span class="cx">
</span><span class="cx"> puts
</span><span class="cx">
</span><del>-def cleanParallelDirectory
</del><ins>+def cleanRunnerDirectory
</ins><span class="cx"> raise unless $bundle
</span><del>- Dir.foreach($parallelDir) {
</del><ins>+ Dir.foreach($runnerDir) {
</ins><span class="cx"> | filename |
</span><span class="cx"> next unless filename =~ /^test_fail/
</span><del>- FileUtils.rm_f $parallelDir + filename
</del><ins>+ FileUtils.rm_f $runnerDir + filename
</ins><span class="cx"> }
</span><span class="cx"> end
</span><span class="cx">
</span><del>-def runParallelTestRunner
- Dir.chdir($parallelDir) {
</del><ins>+def runTestRunner
+ case $testRunnerType
+ when :shell
+ runShellTestRunner
+ when :make
+ runMakeTestRunner
+ else
+ raise "Unknown test runner type: #{$testRunnerType.to_s}"
+ end
+end
+
+def runShellTestRunner
+ Dir.chdir($runnerDir) {
+ mysys("sh", "runscript")
+ }
+end
+
+def runMakeTestRunner
+ Dir.chdir($runnerDir) {
</ins><span class="cx"> # -1 for the Makefile, and -2 for '..' and '.'
</span><span class="cx"> numberOfTests = Dir.entries(".").count - 3
</span><span class="cx"> unless $progressMeter
</span><span class="lines">@@ -1083,7 +1126,7 @@
</span><span class="cx"> def detectFailures
</span><span class="cx"> raise if $bundle
</span><span class="cx">
</span><del>- Dir.foreach($parallelDir) {
</del><ins>+ Dir.foreach($runnerDir) {
</ins><span class="cx"> | filename |
</span><span class="cx"> next unless filename =~ /test_fail_/
</span><span class="cx"> appendFailure($runlist[$~.post_match.to_i])
</span><span class="lines">@@ -1105,13 +1148,16 @@
</span><span class="cx"> end
</span><span class="cx">
</span><span class="cx"> $outputDir = $outputDir.realpath
</span><del>-$parallelDir = $outputDir + ".parallel"
</del><ins>+$runnerDir = $outputDir + ".runner"
</ins><span class="cx">
</span><span class="cx"> prepareBundle unless $bundle
</span><del>-prepareParallelTestRunner unless $bundle
-cleanParallelDirectory if $bundle
</del><ins>+
+puts " "
+
+prepareTestRunner unless $bundle
+cleanRunnerDirectory if $bundle
</ins><span class="cx"> cleanOldResults if $bundle
</span><del>-runParallelTestRunner
</del><ins>+runTestRunner
</ins><span class="cx"> cleanEmptyResultFiles
</span><span class="cx"> detectFailures unless $bundle
</span><span class="cx"> compressBundle if $tarball
</span></span></pre>
</div>
</div>
</body>
</html>