<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[192494] trunk/PerformanceTests</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/192494">192494</a></dd>
<dt>Author</dt> <dd>commit-queue@webkit.org</dd>
<dt>Date</dt> <dd>2015-11-16 15:52:46 -0800 (Mon, 16 Nov 2015)</dd>
</dl>
<h3>Log Message</h3>
<pre>Highlight the alarming test results in the graphics benchmark results page
https://bugs.webkit.org/show_bug.cgi?id=151286
Patch by Said Abou-Hallawa <sabouhallawa@apple.com> on 2015-11-16
Reviewed by Simon Fraser.
When showing the results of a test in the graphics benchmark the following
criteria is going to be applied:
1. If the standard deviation of the test complexity or the frame rate is
equal to or more than 10%, the standard deviation and the test name
will be displayed in red.
2. If the average frame rate is not in the range = [(desired_frame_rate - 2)
.. (desired_frame_rate + 2)], the average frame rate and the test name will
be displayed in red.
* Animometer/resources/extensions.js:
(ResultsTable.prototype._showHeaderRow):
(ResultsTable.prototype._showHeader):
(ResultsTable.prototype._showEmptyCell):
(ResultsTable.prototype._showText):
(ResultsTable.prototype._showFixedNumber):
(ResultsTable.prototype.):
(ResultsTable.prototype._showGraph):
(ResultsTable.prototype._showJSON):
(ResultsTable.prototype._isAlarmingMeasurement):
(ResultsTable.prototype._isAlarmingTestResults):
(ResultsTable.prototype._showEmptyCells):
(ResultsTable.prototype._showEmptyRow):
(ResultsTable.prototype._showTest):
(ResultsTable.prototype._showSuite):
(ResultsTable.prototype._showIteration):
(ResultsTable.prototype.showRecord):
(ResultsTable.prototype.showIterations):
(ResultsTable.prototype._showEmpty): Deleted.
* Animometer/runner/resources/animometer.js:
(window.benchmarkRunnerClient.didFinishLastIteration):
* Animometer/tests/resources/stage.js:
(StageBenchmark.prototype.showResults):</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkPerformanceTestsAnimometerresourcesextensionsjs">trunk/PerformanceTests/Animometer/resources/extensions.js</a></li>
<li><a href="#trunkPerformanceTestsAnimometerrunnerresourcesanimometercss">trunk/PerformanceTests/Animometer/runner/resources/animometer.css</a></li>
<li><a href="#trunkPerformanceTestsAnimometerrunnerresourcesanimometerjs">trunk/PerformanceTests/Animometer/runner/resources/animometer.js</a></li>
<li><a href="#trunkPerformanceTestsAnimometertestsresourcesstagejs">trunk/PerformanceTests/Animometer/tests/resources/stage.js</a></li>
<li><a href="#trunkPerformanceTestsChangeLog">trunk/PerformanceTests/ChangeLog</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkPerformanceTestsAnimometerresourcesextensionsjs"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/Animometer/resources/extensions.js (192493 => 192494)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/Animometer/resources/extensions.js 2015-11-16 23:49:22 UTC (rev 192493)
+++ trunk/PerformanceTests/Animometer/resources/extensions.js 2015-11-16 23:52:46 UTC (rev 192494)
</span><span class="lines">@@ -274,16 +274,14 @@
</span><span class="cx"> _showHeaderRow: function(row, queue, headers, message)
</span><span class="cx"> {
</span><span class="cx"> headers.forEach(function (header) {
</span><del>- var th = document.createElement("th");
</del><ins>+ var th = DocumentExtension.createElement("th", {}, row);
</ins><span class="cx"> th.textContent = header.text;
</span><span class="cx"> if (typeof message != "undefined" && message.length) {
</span><del>- th.appendChild(document.createElement('br'));
- th.appendChild(document.createTextNode('[' + message +']'));
</del><ins>+ th.innerHTML += "<br>" + '[' + message +']';
</ins><span class="cx"> message = "";
</span><span class="cx"> }
</span><span class="cx"> if ("width" in header)
</span><span class="cx"> th.width = header.width + "%";
</span><del>- row.appendChild(th);
</del><span class="cx"> queue.push({element: th, headers: header.children });
</span><span class="cx"> });
</span><span class="cx"> },
</span><span class="lines">@@ -323,37 +321,33 @@
</span><span class="cx"> }
</span><span class="cx"> },
</span><span class="cx">
</span><del>- _showEmpty: function(row)
</del><ins>+ _showEmptyCell: function(row, className)
</ins><span class="cx"> {
</span><del>- var td = document.createElement("td");
- row.appendChild(td);
</del><ins>+ return DocumentExtension.createElement("td", { class: className }, row);
</ins><span class="cx"> },
</span><span class="cx">
</span><del>- _showText: function(row, text)
</del><ins>+ _showText: function(row, text, className)
</ins><span class="cx"> {
</span><del>- var td = document.createElement("td");
</del><ins>+ var td = DocumentExtension.createElement("td", { class: className }, row);
</ins><span class="cx"> td.textContent = text;
</span><del>- row.appendChild(td);
</del><span class="cx"> },
</span><span class="cx">
</span><del>- _showFixedNumber: function(row, value, digits)
</del><ins>+ _showFixedNumber: function(row, value, digits, className)
</ins><span class="cx"> {
</span><del>- var td = document.createElement("td");
</del><ins>+ var td = DocumentExtension.createElement("td", { class: className }, row);
</ins><span class="cx"> td.textContent = value.toFixed(digits || 2);
</span><del>- row.appendChild(td);
</del><span class="cx"> },
</span><span class="cx">
</span><span class="cx"> _showGraph: function(row, testName, testResults)
</span><span class="cx"> {
</span><span class="cx"> var data = testResults[Strings["JSON_SAMPLES"][0]];
</span><span class="cx"> if (!data) {
</span><del>- this._showEmpty(row);
</del><ins>+ this._showEmptyCell(row, "");
</ins><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- var td = document.createElement("td");
- var button = document.createElement("button");
- button.className = "small-button";
</del><ins>+ var td = DocumentExtension.createElement("td", {}, row);
+ var button = DocumentExtension.createElement("button", { class: "small-button" }, td);
</ins><span class="cx">
</span><span class="cx"> button.addEventListener("click", function() {
</span><span class="cx"> var samples = data[Strings["JSON_GRAPH"][0]];
</span><span class="lines">@@ -363,40 +357,78 @@
</span><span class="cx"> });
</span><span class="cx">
</span><span class="cx"> button.textContent = Strings["TEXT_RESULTS"][1] + "...";
</span><del>- td.appendChild(button);
- row.appendChild(td);
</del><span class="cx"> },
</span><span class="cx">
</span><span class="cx"> _showJSON: function(row, testName, testResults)
</span><span class="cx"> {
</span><span class="cx"> var data = testResults[Strings["JSON_SAMPLES"][0]];
</span><span class="cx"> if (!data) {
</span><del>- this._showEmpty(row);
</del><ins>+ this._showEmptyCell(row, "");
</ins><span class="cx"> return;
</span><span class="cx"> }
</span><del>-
- var td = document.createElement("td");
- var button = document.createElement("button");
- button.className = "small-button";
</del><span class="cx">
</span><ins>+ var td = DocumentExtension.createElement("td", {}, row);
+ var button = DocumentExtension.createElement("button", { class: "small-button" }, td);
+
</ins><span class="cx"> button.addEventListener("click", function() {
</span><span class="cx"> benchmarkController.showTestJSON(testName, testResults);
</span><span class="cx"> });
</span><span class="cx">
</span><span class="cx"> button.textContent = Strings["TEXT_RESULTS"][2] + "...";
</span><del>- td.appendChild(button);
- row.appendChild(td);
</del><span class="cx"> },
</span><ins>+
+ _isNoisyMeasurement: function(index, data, measurement, options)
+ {
+ const percentThreshold = 10;
+ const averageThreshold = 2;
+
+ if (measurement == Strings["JSON_MEASUREMENTS"][3])
+ return data[Strings["JSON_MEASUREMENTS"][3]] >= percentThreshold;
+
+ if (index == 1 && measurement == Strings["JSON_MEASUREMENTS"][0])
+ return Math.abs(data[Strings["JSON_MEASUREMENTS"][0]] - options["frame-rate"]) >= averageThreshold;
</ins><span class="cx">
</span><del>- _showTest: function(testName, testResults)
</del><ins>+ return false;
+ },
+
+ _isNoisyTest: function(testResults, options)
</ins><span class="cx"> {
</span><del>- var row = document.createElement("tr");
</del><ins>+ for (var index = 0; index < 2; ++index) {
+ var data = testResults[Strings["JSON_EXPERIMENTS"][index]];
+ for (var measurement in data) {
+ if (this._isNoisyMeasurement(index, data, measurement, options))
+ return true;
+ }
+ }
+ return false;
+ },
+
+ _showEmptyCells: function(row, headers)
+ {
+ for (var index = 0; index < headers.length; ++index) {
+ if (!headers[index].children.length)
+ this._showEmptyCell(row, "suites-separator");
+ else
+ this._showEmptyCells(row, headers[index].children);
+ }
+ },
+
+ _showEmptyRow: function()
+ {
+ var row = DocumentExtension.createElement("tr", {}, this.element);
+ this._showEmptyCells(row, this._headers);
+ },
+
+ _showTest: function(testName, testResults, options)
+ {
+ var row = DocumentExtension.createElement("tr", {}, this.element);
+ var className = this._isNoisyTest(testResults, options) ? "noisy-results" : "";
</ins><span class="cx">
</span><span class="cx"> for (var index = 0; index < this._headers.length; ++index) {
</span><span class="cx">
</span><span class="cx"> switch (index) {
</span><span class="cx"> case 0:
</span><del>- this._showText(row, testName);
</del><ins>+ this._showText(row, testName, className);
</ins><span class="cx"> break;
</span><span class="cx">
</span><span class="cx"> case 1:
</span><span class="lines">@@ -408,7 +440,7 @@
</span><span class="cx"> case 3:
</span><span class="cx"> var data = testResults[Strings["JSON_EXPERIMENTS"][index - 2]];
</span><span class="cx"> for (var measurement in data)
</span><del>- this._showFixedNumber(row, data[measurement], 2);
</del><ins>+ this._showFixedNumber(row, data[measurement], 2, this._isNoisyMeasurement(index - 2, data, measurement, options) ? className : "");
</ins><span class="cx"> break;
</span><span class="cx">
</span><span class="cx"> case 4:
</span><span class="lines">@@ -417,38 +449,38 @@
</span><span class="cx"> break;
</span><span class="cx"> }
</span><span class="cx"> }
</span><del>-
- this.element.appendChild(row);
</del><span class="cx"> },
</span><span class="cx">
</span><del>- _showSuite: function(suiteName, suiteResults)
</del><ins>+ _showSuite: function(suiteName, suiteResults, options)
</ins><span class="cx"> {
</span><span class="cx"> for (var testName in suiteResults[Strings["JSON_RESULTS"][2]]) {
</span><del>- this._showTest(testName, suiteResults[Strings["JSON_RESULTS"][2]][testName]);
</del><ins>+ this._showTest(testName, suiteResults[Strings["JSON_RESULTS"][2]][testName], options);
</ins><span class="cx"> }
</span><span class="cx"> },
</span><span class="cx">
</span><del>- _showIteration : function(iterationResults)
</del><ins>+ _showIteration : function(iterationResults, options)
</ins><span class="cx"> {
</span><span class="cx"> for (var suiteName in iterationResults[Strings["JSON_RESULTS"][1]]) {
</span><del>- this._showSuite(suiteName, iterationResults[Strings["JSON_RESULTS"][1]][suiteName]);
</del><ins>+ if (suiteName != Object.keys(iterationResults[Strings["JSON_RESULTS"][1]])[0])
+ this._showEmptyRow();
+ this._showSuite(suiteName, iterationResults[Strings["JSON_RESULTS"][1]][suiteName], options);
</ins><span class="cx"> }
</span><span class="cx"> },
</span><span class="cx">
</span><del>- showRecord: function(testName, message, testResults)
</del><ins>+ showRecord: function(testName, message, testResults, options)
</ins><span class="cx"> {
</span><span class="cx"> this.clear();
</span><span class="cx"> this._showHeader(message);
</span><del>- this._showTest(testName, testResults);
</del><ins>+ this._showTest(testName, testResults, options);
</ins><span class="cx"> },
</span><span class="cx">
</span><del>- showIterations: function(iterationsResults)
</del><ins>+ showIterations: function(iterationsResults, options)
</ins><span class="cx"> {
</span><span class="cx"> this.clear();
</span><span class="cx"> this._showHeader("");
</span><span class="cx">
</span><span class="cx"> iterationsResults.forEach(function(iterationResults) {
</span><del>- this._showIteration(iterationResults);
</del><ins>+ this._showIteration(iterationResults, options);
</ins><span class="cx"> }, this);
</span><span class="cx"> }
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkPerformanceTestsAnimometerrunnerresourcesanimometercss"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/Animometer/runner/resources/animometer.css (192493 => 192494)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/Animometer/runner/resources/animometer.css 2015-11-16 23:49:22 UTC (rev 192493)
+++ trunk/PerformanceTests/Animometer/runner/resources/animometer.css 2015-11-16 23:52:46 UTC (rev 192494)
</span><span class="lines">@@ -161,6 +161,14 @@
</span><span class="cx"> color: black;
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+.results-table td.noisy-results {
+ color: red;
+}
+
+.results-table td.suites-separator {
+ background-color: yellow;
+}
+
</ins><span class="cx"> /* -------------------------------------------------------------------------- */
</span><span class="cx"> /* Results JSON */
</span><span class="cx"> /* -------------------------------------------------------------------------- */
</span></span></pre></div>
<a id="trunkPerformanceTestsAnimometerrunnerresourcesanimometerjs"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/Animometer/runner/resources/animometer.js (192493 => 192494)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/Animometer/runner/resources/animometer.js 2015-11-16 23:49:22 UTC (rev 192493)
+++ trunk/PerformanceTests/Animometer/runner/resources/animometer.js 2015-11-16 23:52:46 UTC (rev 192494)
</span><span class="lines">@@ -45,7 +45,7 @@
</span><span class="cx"> {
</span><span class="cx"> var json = this._resultsDashboard.toJSON(true, true);
</span><span class="cx"> this.score = json[Strings["JSON_SCORE"]];
</span><del>- this._resultsTable.showIterations(json[Strings["JSON_RESULTS"][0]]);
</del><ins>+ this._resultsTable.showIterations(json[Strings["JSON_RESULTS"][0]], this.options);
</ins><span class="cx"> sectionsManager.showJSON("json", json[Strings["JSON_RESULTS"][0]][0]);
</span><span class="cx"> suitesManager.updateLocalStorageFromJSON(json[Strings["JSON_RESULTS"][0]][0]);
</span><span class="cx"> benchmarkController.showResults();
</span></span></pre></div>
<a id="trunkPerformanceTestsAnimometertestsresourcesstagejs"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/Animometer/tests/resources/stage.js (192493 => 192494)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/Animometer/tests/resources/stage.js 2015-11-16 23:49:22 UTC (rev 192493)
+++ trunk/PerformanceTests/Animometer/tests/resources/stage.js 2015-11-16 23:52:46 UTC (rev 192494)
</span><span class="lines">@@ -173,7 +173,7 @@
</span><span class="cx"> return;
</span><span class="cx">
</span><span class="cx"> if (this._options["show-running-results"])
</span><del>- this._recordTable.showRecord(this._test.name, message, this._sampler.toJSON(true, false));
</del><ins>+ this._recordTable.showRecord(this._test.name, message, this._sampler.toJSON(true, false), this._options);
</ins><span class="cx">
</span><span class="cx"> this._progressBar.setPos(progress);
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkPerformanceTestsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/ChangeLog (192493 => 192494)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/ChangeLog 2015-11-16 23:49:22 UTC (rev 192493)
+++ trunk/PerformanceTests/ChangeLog 2015-11-16 23:52:46 UTC (rev 192494)
</span><span class="lines">@@ -1,5 +1,46 @@
</span><span class="cx"> 2015-11-16 Said Abou-Hallawa <sabouhallawa@apple.com>
</span><span class="cx">
</span><ins>+ Highlight the alarming test results in the graphics benchmark results page
+ https://bugs.webkit.org/show_bug.cgi?id=151286
+
+ Reviewed by Simon Fraser.
+
+ When showing the results of a test in the graphics benchmark the following
+ criteria is going to be applied:
+
+ 1. If the standard deviation of the test complexity or the frame rate is
+ equal to or more than 10%, the standard deviation and the test name
+ will be displayed in red.
+ 2. If the average frame rate is not in the range = [(desired_frame_rate - 2)
+ .. (desired_frame_rate + 2)], the average frame rate and the test name will
+ be displayed in red.
+
+ * Animometer/resources/extensions.js:
+ (ResultsTable.prototype._showHeaderRow):
+ (ResultsTable.prototype._showHeader):
+ (ResultsTable.prototype._showEmptyCell):
+ (ResultsTable.prototype._showText):
+ (ResultsTable.prototype._showFixedNumber):
+ (ResultsTable.prototype.):
+ (ResultsTable.prototype._showGraph):
+ (ResultsTable.prototype._showJSON):
+ (ResultsTable.prototype._isAlarmingMeasurement):
+ (ResultsTable.prototype._isAlarmingTestResults):
+ (ResultsTable.prototype._showEmptyCells):
+ (ResultsTable.prototype._showEmptyRow):
+ (ResultsTable.prototype._showTest):
+ (ResultsTable.prototype._showSuite):
+ (ResultsTable.prototype._showIteration):
+ (ResultsTable.prototype.showRecord):
+ (ResultsTable.prototype.showIterations):
+ (ResultsTable.prototype._showEmpty): Deleted.
+ * Animometer/runner/resources/animometer.js:
+ (window.benchmarkRunnerClient.didFinishLastIteration):
+ * Animometer/tests/resources/stage.js:
+ (StageBenchmark.prototype.showResults):
+
+2015-11-16 Said Abou-Hallawa <sabouhallawa@apple.com>
+
</ins><span class="cx"> Clean referencing the options object in the graphics benchmark
</span><span class="cx"> https://bugs.webkit.org/show_bug.cgi?id=151284
</span><span class="cx">
</span></span></pre>
</div>
</div>
</body>
</html>