<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[168732] trunk/PerformanceTests</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/168732">168732</a></dd>
<dt>Author</dt> <dd>rniwa@webkit.org</dd>
<dt>Date</dt> <dd>2014-05-13 14:08:10 -0700 (Tue, 13 May 2014)</dd>
</dl>
<h3>Log Message</h3>
<pre>DYEBench should run 20 iterations in browser
https://bugs.webkit.org/show_bug.cgi?id=132795
Reviewed by Gavin Barraclough.
Right now, run-perf-tests runs BYEBench 4 times with different instances of DRT/WTR to get
a more stable time and account for differences in the runtime environment, particularly,
ASLR (Address Space Layout Randomization).
While we can't account for the latter effect when the benchmark is ran inside a browser,
we can at least run 20 iterations to get a more stable (but perhaps biased by ALSR) number.
While the sample size of 20 is statistically unsound to compute the arithmetic mean from,
it's MUCH better than the current sample size of 5.
* DoYouEvenBench/Full.html:
(benchmarkClient.iterationCount): Set the default iteration count to 20.
(startTest): Use benchmarkClient.iterationCount as the iteration count.
* DoYouEvenBench/resources/benchmark-report.js:
(benchmarkClient.iterationCount): Continue to use the old iteration count of 5 when ran
inside run-perf-tests.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkPerformanceTestsChangeLog">trunk/PerformanceTests/ChangeLog</a></li>
<li><a href="#trunkPerformanceTestsDoYouEvenBenchFullhtml">trunk/PerformanceTests/DoYouEvenBench/Full.html</a></li>
<li><a href="#trunkPerformanceTestsDoYouEvenBenchresourcesbenchmarkreportjs">trunk/PerformanceTests/DoYouEvenBench/resources/benchmark-report.js</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkPerformanceTestsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/ChangeLog (168731 => 168732)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/ChangeLog 2014-05-13 21:07:22 UTC (rev 168731)
+++ trunk/PerformanceTests/ChangeLog 2014-05-13 21:08:10 UTC (rev 168732)
</span><span class="lines">@@ -1,3 +1,27 @@
</span><ins>+2014-05-13 Ryosuke Niwa <rniwa@webkit.org>
+
+ DYEBench should run 20 iterations in browser
+ https://bugs.webkit.org/show_bug.cgi?id=132795
+
+ Reviewed by Gavin Barraclough.
+
+ Right now, run-perf-tests runs BYEBench 4 times with different instances of DRT/WTR to get
+ a more stable time and account for differences in the runtime environment, particularly,
+ ASLR (Address Space Layout Randomization).
+
+ While we can't account for the latter effect when the benchmark is ran inside a browser,
+ we can at least run 20 iterations to get a more stable (but perhaps biased by ALSR) number.
+
+ While the sample size of 20 is statistically unsound to compute the arithmetic mean from,
+ it's MUCH better than the current sample size of 5.
+
+ * DoYouEvenBench/Full.html:
+ (benchmarkClient.iterationCount): Set the default iteration count to 20.
+ (startTest): Use benchmarkClient.iterationCount as the iteration count.
+ * DoYouEvenBench/resources/benchmark-report.js:
+ (benchmarkClient.iterationCount): Continue to use the old iteration count of 5 when ran
+ inside run-perf-tests.
+
</ins><span class="cx"> 2014-05-09 Ryosuke Niwa <rniwa@webkit.org>
</span><span class="cx">
</span><span class="cx"> DYEBench should show 95th percentile right next to the mean with ±
</span></span></pre></div>
<a id="trunkPerformanceTestsDoYouEvenBenchFullhtml"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/DoYouEvenBench/Full.html (168731 => 168732)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/DoYouEvenBench/Full.html 2014-05-13 21:07:22 UTC (rev 168731)
+++ trunk/PerformanceTests/DoYouEvenBench/Full.html 2014-05-13 21:08:10 UTC (rev 168732)
</span><span class="lines">@@ -1,7 +1,7 @@
</span><span class="cx"> <!DOCTYPE html>
</span><span class="cx"> <html>
</span><span class="cx"> <head>
</span><del>-<title>DoYouEvenBench v0.10</title>
</del><ins>+<title>DoYouEvenBench v0.11</title>
</ins><span class="cx"> <style type="text/css">
</span><span class="cx"> caption { margin: 0; padding: 0; font-family: sans-serif; font-size: 1em; font-weight: bold; white-space: nowrap; }
</span><span class="cx"> #progressContainer { padding: 605px 0 10px 0; width: 800px; }
</span><span class="lines">@@ -43,6 +43,7 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> window.benchmarkClient = {
</span><ins>+ iterationCount: 20,
</ins><span class="cx"> willRunTest: function () {
</span><span class="cx"> if (!progress) {
</span><span class="cx"> // We don't use the real progress element as some implementations animate it.
</span><span class="lines">@@ -79,10 +80,10 @@
</span><span class="cx"> })();
</span><span class="cx">
</span><span class="cx"> function startTest() {
</span><del>- var iterationCount = 5;
- benchmarkClient.testsCount = iterationCount * Suites.reduce(function (testsCount, suite) { return testsCount + suite.tests.length; }, 0);
</del><ins>+ var totalSubtestCount = Suites.reduce(function (testsCount, suite) { return testsCount + suite.tests.length; }, 0);
+ benchmarkClient.testsCount = benchmarkClient.iterationCount * totalSubtestCount;
</ins><span class="cx"> var runner = new BenchmarkRunner(Suites, benchmarkClient);
</span><del>- runner.runMultipleIterations(iterationCount);
</del><ins>+ runner.runMultipleIterations(benchmarkClient.iterationCount);
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> </script>
</span></span></pre></div>
<a id="trunkPerformanceTestsDoYouEvenBenchresourcesbenchmarkreportjs"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/DoYouEvenBench/resources/benchmark-report.js (168731 => 168732)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/DoYouEvenBench/resources/benchmark-report.js 2014-05-13 21:07:22 UTC (rev 168731)
+++ trunk/PerformanceTests/DoYouEvenBench/resources/benchmark-report.js 2014-05-13 21:08:10 UTC (rev 168732)
</span><span class="lines">@@ -19,6 +19,7 @@
</span><span class="cx"> var valuesByIteration = new Array;
</span><span class="cx">
</span><span class="cx"> window.benchmarkClient = {
</span><ins>+ iterationCount: 5, // Use 4 different instances of DRT/WTR to run 5 iterations.
</ins><span class="cx"> willStartFirstIteration: function (iterationCount) {
</span><span class="cx"> createTest = function (name, aggregator, isLastTest) {
</span><span class="cx"> return {
</span></span></pre>
</div>
</div>
</body>
</html>