<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[197361] trunk/Tools</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/197361">197361</a></dd>
<dt>Author</dt> <dd>fpizlo@apple.com</dd>
<dt>Date</dt> <dd>2016-02-29 11:35:17 -0800 (Mon, 29 Feb 2016)</dd>
</dl>
<h3>Log Message</h3>
<pre>We've been running Octane/regexp all wrong in run-jsc-benchmarks
https://bugs.webkit.org/show_bug.cgi?id=154827
Reviewed by Andreas Kling.
Octane v.2 and JetStream v.1.1 run this benchmark with warmup. This script was running
it without warmup. This patches fixes this by making this script run it with warmup.
This fix shows that my last patch, which added FTL support for regexp, was actually a 3%
speed-up on Octane/regexp, not a slow-down as the ChangeLog claimed.
It discovered this bug because for each Octane test that I want to debug, I usually make
a standalone .js file that contains the whole test along with a miniharness - usually
a plain loop - that runs it almost like it would for real but with whatever hacks I'm
using for debugging. When I wrote such a thing for regexp, I used a ~20 iteration warmup
to match the one second of warmup that this benchmark gets in Octane. To my surprise,
this quite faithful regexp runner did not see the regression that run-jsc-benchmarks
saw. That's when I found out that run-jsc-benchmarks was running it wrong.
The reason for the no-warmup slow-down is that the FTL is actually fairly expensive to
run on some of these very large functions in the regexp benchmark. I don't think we can
do anything about that, and I'd argue that the speed-up we see after the compilation is
done suggests that it was worth it.
* Scripts/run-jsc-benchmarks:</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkToolsChangeLog">trunk/Tools/ChangeLog</a></li>
<li><a href="#trunkToolsScriptsrunjscbenchmarks">trunk/Tools/Scripts/run-jsc-benchmarks</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkToolsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Tools/ChangeLog (197360 => 197361)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/ChangeLog 2016-02-29 18:55:21 UTC (rev 197360)
+++ trunk/Tools/ChangeLog 2016-02-29 19:35:17 UTC (rev 197361)
</span><span class="lines">@@ -1,3 +1,31 @@
</span><ins>+2016-02-29 Filip Pizlo <fpizlo@apple.com>
+
+ We've been running Octane/regexp all wrong in run-jsc-benchmarks
+ https://bugs.webkit.org/show_bug.cgi?id=154827
+
+ Reviewed by Andreas Kling.
+
+ Octane v.2 and JetStream v.1.1 run this benchmark with warmup. This script was running
+ it without warmup. This patches fixes this by making this script run it with warmup.
+
+ This fix shows that my last patch, which added FTL support for regexp, was actually a 3%
+ speed-up on Octane/regexp, not a slow-down as the ChangeLog claimed.
+
+ It discovered this bug because for each Octane test that I want to debug, I usually make
+ a standalone .js file that contains the whole test along with a miniharness - usually
+ a plain loop - that runs it almost like it would for real but with whatever hacks I'm
+ using for debugging. When I wrote such a thing for regexp, I used a ~20 iteration warmup
+ to match the one second of warmup that this benchmark gets in Octane. To my surprise,
+ this quite faithful regexp runner did not see the regression that run-jsc-benchmarks
+ saw. That's when I found out that run-jsc-benchmarks was running it wrong.
+
+ The reason for the no-warmup slow-down is that the FTL is actually fairly expensive to
+ run on some of these very large functions in the regexp benchmark. I don't think we can
+ do anything about that, and I'd argue that the speed-up we see after the compilation is
+ done suggests that it was worth it.
+
+ * Scripts/run-jsc-benchmarks:
+
</ins><span class="cx"> 2016-02-29 Gavin Barraclough <barraclough@apple.com>
</span><span class="cx">
</span><span class="cx"> RefCounter<T>::Event -> RefCounterEvent
</span></span></pre></div>
<a id="trunkToolsScriptsrunjscbenchmarks"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/run-jsc-benchmarks (197360 => 197361)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/run-jsc-benchmarks 2016-02-29 18:55:21 UTC (rev 197360)
+++ trunk/Tools/Scripts/run-jsc-benchmarks 2016-02-29 19:35:17 UTC (rev 197361)
</span><span class="lines">@@ -2930,7 +2930,7 @@
</span><span class="cx"> [["raytrace"], "raytrace", 2, true, false, 32],
</span><span class="cx"> [["richards"], "richards", 2, true, false, 32],
</span><span class="cx"> [["splay"], "splay", 2, true, false, 32],
</span><del>- [["regexp"], "regexp", 2, false, false, 16],
</del><ins>+ [["regexp"], "regexp", 2, true, false, 16],
</ins><span class="cx"> [["pdfjs"], "pdfjs", 2, false, false, 4],
</span><span class="cx"> [["mandreel"], "mandreel", 2, false, false, 4],
</span><span class="cx"> [["gbemu-part1", "gbemu-part2"], "gbemu", 2, false, false, 4],
</span></span></pre>
</div>
</div>
</body>
</html>