<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[184806] trunk/Websites/perf.webkit.org</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/184806">184806</a></dd>
<dt>Author</dt> <dd>rniwa@webkit.org</dd>
<dt>Date</dt> <dd>2015-05-22 17:25:31 -0700 (Fri, 22 May 2015)</dd>
</dl>
<h3>Log Message</h3>
<pre>Show results and status before revisions for A/B testing results
https://bugs.webkit.org/show_bug.cgi?id=145327
Reviewed by Chris Dumez.
Place the results and the status columns before the columns for revisions.
Also show the absolute difference as well as the relative difference between the averages of A and B.
* public/v2/app.js:
(App.TestGroupPane._populate):
(App.TestGroupPane._computeStatisticalSignificance):
* public/v2/index.html:</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkWebsitesperfwebkitorgChangeLog">trunk/Websites/perf.webkit.org/ChangeLog</a></li>
<li><a href="#trunkWebsitesperfwebkitorgpublicv2appjs">trunk/Websites/perf.webkit.org/public/v2/app.js</a></li>
<li><a href="#trunkWebsitesperfwebkitorgpublicv2indexhtml">trunk/Websites/perf.webkit.org/public/v2/index.html</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkWebsitesperfwebkitorgChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Websites/perf.webkit.org/ChangeLog (184805 => 184806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/ChangeLog 2015-05-23 00:19:03 UTC (rev 184805)
+++ trunk/Websites/perf.webkit.org/ChangeLog 2015-05-23 00:25:31 UTC (rev 184806)
</span><span class="lines">@@ -1,3 +1,18 @@
</span><ins>+2015-05-22 Ryosuke Niwa <rniwa@webkit.org>
+
+ Show results and status before revisions for A/B testing results
+ https://bugs.webkit.org/show_bug.cgi?id=145327
+
+ Reviewed by Chris Dumez.
+
+ Place the results and the status columns before the columns for revisions.
+ Also show the absolute difference as well as the relative difference between the averages of A and B.
+
+ * public/v2/app.js:
+ (App.TestGroupPane._populate):
+ (App.TestGroupPane._computeStatisticalSignificance):
+ * public/v2/index.html:
+
</ins><span class="cx"> 2015-05-20 Ryosuke Niwa <rniwa@webkit.org>
</span><span class="cx">
</span><span class="cx"> Build fix after r184591.
</span></span></pre></div>
<a id="trunkWebsitesperfwebkitorgpublicv2appjs"></a>
<div class="modfile"><h4>Modified: trunk/Websites/perf.webkit.org/public/v2/app.js (184805 => 184806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/public/v2/app.js 2015-05-23 00:19:03 UTC (rev 184805)
+++ trunk/Websites/perf.webkit.org/public/v2/app.js 2015-05-23 00:25:31 UTC (rev 184806)
</span><span class="lines">@@ -1424,14 +1424,21 @@
</span><span class="cx">
</span><span class="cx"> this.set('configurations', configurations);
</span><span class="cx">
</span><ins>+ var probabilityFormatter = d3.format('.2p');
</ins><span class="cx"> var comparisons = [];
</span><span class="cx"> for (var i = 0; i < configurations.length - 1; i++) {
</span><span class="cx"> var summary1 = configurations[i].summary;
</span><span class="cx"> for (var j = i + 1; j < configurations.length; j++) {
</span><span class="cx"> var summary2 = configurations[j].summary;
</span><ins>+
+ var valueDelta = testResults.deltaFormatter(summary2.value - summary1.value);
+ var relativeDelta = d3.format('+.2p')((summary2.value - summary1.value) / summary1.value);
+
+ var stat = this._computeStatisticalSignificance(summary1.measuredValues, summary2.measuredValues);
</ins><span class="cx"> comparisons.push({
</span><span class="cx"> label: summary1.configLetter + ' / ' + summary2.configLetter,
</span><del>- result: this._computeStatisticalSignificance(summary1.measuredValues, summary2.measuredValues)
</del><ins>+ difference: isNaN(summary1.value) || isNaN(summary2.value) ? 'N/A' : valueDelta + ' (' + relativeDelta + ') ',
+ result: stat,
</ins><span class="cx"> });
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="lines">@@ -1448,13 +1455,13 @@
</span><span class="cx"> var details = ' (t=' + tFormatter(statistics.t) + ' df=' + tFormatter(statistics.degreesOfFreedom) + ')';
</span><span class="cx">
</span><span class="cx"> if (!statistics.range[0])
</span><del>- return 'Not statistically significant' + details;
</del><ins>+ return 'Not significant' + details;
</ins><span class="cx">
</span><span class="cx"> var lowerLimit = probabilityFormatter(statistics.range[0]);
</span><span class="cx"> if (!statistics.range[1])
</span><del>- return 'Statistical significance > ' + lowerLimit + details;
</del><ins>+ return 'Significance > ' + lowerLimit + details;
</ins><span class="cx">
</span><del>- return lowerLimit + ' < Statistical significance < ' + probabilityFormatter(statistics.range[1]) + details;
</del><ins>+ return lowerLimit + ' < Significance < ' + probabilityFormatter(statistics.range[1]) + details;
</ins><span class="cx"> },
</span><span class="cx"> _updateReferenceChart: function ()
</span><span class="cx"> {
</span></span></pre></div>
<a id="trunkWebsitesperfwebkitorgpublicv2indexhtml"></a>
<div class="modfile"><h4>Modified: trunk/Websites/perf.webkit.org/public/v2/index.html (184805 => 184806)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/public/v2/index.html 2015-05-23 00:19:03 UTC (rev 184805)
+++ trunk/Websites/perf.webkit.org/public/v2/index.html 2015-05-23 00:25:31 UTC (rev 184806)
</span><span class="lines">@@ -628,11 +628,11 @@
</span><span class="cx"> <thead>
</span><span class="cx"> <tr>
</span><span class="cx"> <td colspan="2">Configuration</td>
</span><ins>+ <td>Results</td>
+ <td>Status</td>
</ins><span class="cx"> {{#each repositories}}
</span><span class="cx"> <td>{{name}}</td>
</span><span class="cx"> {{/each}}
</span><del>- <td>Results</td>
- <td>Status</td>
</del><span class="cx"> </tr>
</span><span class="cx"> </thead>
</span><span class="cx"> {{#each configurations}}
</span><span class="lines">@@ -658,12 +658,13 @@
</span><span class="cx"> <tbody>
</span><span class="cx"> <tr>
</span><span class="cx"> <td colspan="2">{{label}}</td>
</span><ins>+ <td>{{difference}}</td>
+ <td>{{result}}</td>
</ins><span class="cx"> {{#with ../this}}
</span><span class="cx"> {{#each repositories}}
</span><span class="cx"> <td></td>
</span><span class="cx"> {{/each}}
</span><span class="cx"> {{/with}}
</span><del>- <td colspan="2">{{result}}</td>
</del><span class="cx"> </tr>
</span><span class="cx"> </tbody>
</span><span class="cx"> {{/each}}
</span><span class="lines">@@ -684,18 +685,18 @@
</span><span class="cx"> </script>
</span><span class="cx">
</span><span class="cx"> <script type="text/x-handlebars" data-template-name="testGroupRow">
</span><del>- {{#each revisionList}}
- <td>{{this}}</td>
- {{/each}}
</del><span class="cx"> <td>
</span><span class="cx"> {{#if value}}
</span><del>- {{box-plot range=valueRange value=value delta=confidenceIntervalDelta}}
</del><ins>+ {{box-plot range=valueRange value=value delta=delta}}
</ins><span class="cx"> {{/if}}
</span><span class="cx"> {{formattedValue}}
</span><span class="cx"> </td>
</span><span class="cx"> <td>
</span><span class="cx"> <a {{bind-attr href=url title=buildLabel}}>{{statusLabel}}</a>
</span><span class="cx"> </td>
</span><ins>+ {{#each revisionList}}
+ <td>{{this}}</td>
+ {{/each}}
</ins><span class="cx"> </script>
</span><span class="cx">
</span><span class="cx"> <script type="text/x-handlebars" data-template-name="testGroupForm">
</span></span></pre>
</div>
</div>
</body>
</html>