<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[191873] trunk/PerformanceTests</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/191873">191873</a></dd>
<dt>Author</dt> <dd>commit-queue@webkit.org</dd>
<dt>Date</dt> <dd>2015-11-01 18:31:02 -0800 (Sun, 01 Nov 2015)</dd>
</dl>
<h3>Log Message</h3>
<pre>Add an option to make the graphics benchmark runs a specific test with fixed complexity
https://bugs.webkit.org/show_bug.cgi?id=150529
Patch by Said Abou-Hallawa <sabouhallawa@apple,com> on 2015-11-01
Reviewed by Darin Adler.
Beside each test in the suites tree, we are going to show the complexity
arithmetic mean of the of the last run in an edit control. Based on a
new option these edit controls will all be visible or hidden. If they are
visible their values can be changed. The benchmark runner if it run in
the non-adaptive mode will set the complexity of the test to the passed
value and will not change it ever. The animator will animate the test and
frame rate will also be measured.
* Animometer/runner/animometer.html: Add a new option for the non-adaptive mode.
* Animometer/runner/resources/animometer.css:
(section#home input[type="number"]): Define the width of all the edit control in the <home> section.
(section#home > suites input[type="number"]): The edit controls in the <suites> box will be right aligned and hidden by default.
(section#home > suites input[type="number"].selected): When the class "selected" is added, the edit controls will be visible.
(section#home > options > label > input[type="number"]): Deleted.
* Animometer/runner/resources/animometer.js:
(window.benchmarkRunnerClient.didFinishLastIteration): Update the local storage with the results of each test.
(window.optionsManager._adaptiveTestElement): Returns the checkbox for setting the adaptive test option.
(window.suitesManager._editElement): Returns the edit element associated with each test element in the suites tree.
(window.suitesManager._editsElements): Returns a list of all the elements in the <suites> box.
(window.suitesManager._localStorageNameForTest): Change this function to take strings.
(window.suitesManager._createTestElement): Adds an edit control beside each test.
(window.suitesManager.updateEditsElementsState): Adds/Removes the 'selected' class to/from all the tests edit elements.
(window.suitesManager.updateUIFromLocalStorage): Reads the edit control value from the local storage.
(window.suitesManager.updateLocalStorageFromUI): Saves the edit control value to the local storage.
(window.suitesManager.updateLocalStorageFromJSON): Saves the last run results to the local storage.
(window.benchmarkController.initialize): Shows/Hides the test edit controls based on the adaptive test option.
(window.benchmarkController.onChangeAdaptiveTestCheckbox): An onchange event handler for the adaptive test checkbox.
* Animometer/tests/resources/main.js:
(Benchmark.prototype.update): Fix the complexity of the test if the running mode is non-adaptive test and desired complexity is not zero.
(window.runBenchmark): Add the test complexity as a new benchmark option.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkPerformanceTestsAnimometerrunneranimometerhtml">trunk/PerformanceTests/Animometer/runner/animometer.html</a></li>
<li><a href="#trunkPerformanceTestsAnimometerrunnerresourcesanimometercss">trunk/PerformanceTests/Animometer/runner/resources/animometer.css</a></li>
<li><a href="#trunkPerformanceTestsAnimometerrunnerresourcesanimometerjs">trunk/PerformanceTests/Animometer/runner/resources/animometer.js</a></li>
<li><a href="#trunkPerformanceTestsAnimometertestsresourcesmainjs">trunk/PerformanceTests/Animometer/tests/resources/main.js</a></li>
<li><a href="#trunkPerformanceTestsChangeLog">trunk/PerformanceTests/ChangeLog</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkPerformanceTestsAnimometerrunneranimometerhtml"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/Animometer/runner/animometer.html (191872 => 191873)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/Animometer/runner/animometer.html 2015-11-02 01:01:28 UTC (rev 191872)
+++ trunk/PerformanceTests/Animometer/runner/animometer.html 2015-11-02 02:31:02 UTC (rev 191873)
</span><span class="lines">@@ -32,7 +32,8 @@
</span><span class="cx"> <label><input id="estimated-frame-rate" type="checkbox" checked> Estimated Frame Rate</label><br>
</span><span class="cx"> <label><input id="fix-test-complexity" type="checkbox"> Fix test complexity after warmup</label><br>
</span><span class="cx"> <label><input id="show-running-results" type="checkbox"> Show running results</label><br>
</span><del>- <label><input id="normalize-for-device-scale-factor" type="checkbox"> Normalize for device scale factor</label>
</del><ins>+ <label><input id="normalize-for-device-scale-factor" type="checkbox"> Normalize for device scale factor</label><br>
+ <label><input id="adaptive-test" type="checkbox" checked onchange="benchmarkController.onChangeAdaptiveTestCheckbox()"> Adaptive test</label>
</ins><span class="cx"> </options>
</span><span class="cx"> <footer>
</span><span class="cx"> <button class="large-button" onclick="benchmarkController.startTest()">Start Test</button>
</span></span></pre></div>
<a id="trunkPerformanceTestsAnimometerrunnerresourcesanimometercss"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/Animometer/runner/resources/animometer.css (191872 => 191873)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/Animometer/runner/resources/animometer.css 2015-11-02 01:01:28 UTC (rev 191872)
+++ trunk/PerformanceTests/Animometer/runner/resources/animometer.css 2015-11-02 02:31:02 UTC (rev 191873)
</span><span class="lines">@@ -279,10 +279,19 @@
</span><span class="cx"> text-align: center;
</span><span class="cx"> }
</span><span class="cx">
</span><del>-section#home > options > label > input[type="number"] {
- width: 50px;
</del><ins>+section#home input[type="number"] {
+ width: 70px;
</ins><span class="cx"> }
</span><ins>+
+section#home > suites input[type="number"] {
+ display: none;
+ float: right;
+}
</ins><span class="cx">
</span><ins>+section#home > suites input[type="number"].selected {
+ display: inline;
+}
+
</ins><span class="cx"> /* -------------------------------------------------------------------------- */
</span><span class="cx"> /* Running Section */
</span><span class="cx"> /* -------------------------------------------------------------------------- */
</span></span></pre></div>
<a id="trunkPerformanceTestsAnimometerrunnerresourcesanimometerjs"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/Animometer/runner/resources/animometer.js (191872 => 191873)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/Animometer/runner/resources/animometer.js 2015-11-02 01:01:28 UTC (rev 191872)
+++ trunk/PerformanceTests/Animometer/runner/resources/animometer.js 2015-11-02 02:31:02 UTC (rev 191873)
</span><span class="lines">@@ -47,6 +47,7 @@
</span><span class="cx"> this.score = json[Strings["JSON_SCORE"]];
</span><span class="cx"> this._resultsTable.showIterations(json[Strings["JSON_RESULTS"][0]]);
</span><span class="cx"> sectionsManager.showJSON("json", json[Strings["JSON_RESULTS"][0]][0]);
</span><ins>+ suitesManager.updateLocalStorageFromJSON(json[Strings["JSON_RESULTS"][0]][0]);
</ins><span class="cx"> benchmarkController.showResults();
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="lines">@@ -142,6 +143,11 @@
</span><span class="cx"> return document.querySelectorAll("section#home > options input");;
</span><span class="cx"> },
</span><span class="cx">
</span><ins>+ _adaptiveTestElement: function()
+ {
+ return document.querySelector("section#home > options #adaptive-test");;
+ },
+
</ins><span class="cx"> updateUIFromLocalStorage: function()
</span><span class="cx"> {
</span><span class="cx"> var optionsElements = this._optionsElements();
</span><span class="lines">@@ -197,11 +203,21 @@
</span><span class="cx"> return element.querySelector("input[type='checkbox']:not(.expand-button)");
</span><span class="cx"> },
</span><span class="cx">
</span><del>- _localStorageNameForTest: function(suite, test)
</del><ins>+ _editElement: function(element)
</ins><span class="cx"> {
</span><del>- return suite.name + "/" + test.name;
</del><ins>+ return element.querySelector("input[type='number']");
</ins><span class="cx"> },
</span><span class="cx">
</span><ins>+ _editsElements: function()
+ {
+ return document.querySelectorAll("section#home > suites input[type='number']");
+ },
+
+ _localStorageNameForTest: function(suiteName, testName)
+ {
+ return suiteName + "/" + testName;
+ },
+
</ins><span class="cx"> _updateSuiteCheckboxState: function(suiteCheckbox)
</span><span class="cx"> {
</span><span class="cx"> var numberEnabledTests = 0;
</span><span class="lines">@@ -276,6 +292,7 @@
</span><span class="cx">
</span><span class="cx"> suiteCheckbox.testsElements.push(testElement);
</span><span class="cx"> span.appendChild(document.createTextNode(" " + test.name));
</span><ins>+ DocumentExtension.createElement("input", { type: "number" }, testElement);
</ins><span class="cx"> return testElement;
</span><span class="cx"> },
</span><span class="cx">
</span><span class="lines">@@ -294,6 +311,20 @@
</span><span class="cx"> }, this);
</span><span class="cx"> },
</span><span class="cx">
</span><ins>+ updateEditsElementsState: function()
+ {
+ var editsElements = this._editsElements();
+ var show = !optionsManager._adaptiveTestElement().checked;
+
+ for (var i = 0; i < editsElements.length; ++i) {
+ var editElement = editsElements[i];
+ if (show)
+ editElement.classList.add("selected");
+ else
+ editElement.classList.remove("selected");
+ }
+ },
+
</ins><span class="cx"> updateUIFromLocalStorage: function()
</span><span class="cx"> {
</span><span class="cx"> var suitesElements = this._suitesElements();
</span><span class="lines">@@ -305,14 +336,16 @@
</span><span class="cx">
</span><span class="cx"> suiteCheckbox.testsElements.forEach(function(testElement) {
</span><span class="cx"> var testCheckbox = this._checkboxElement(testElement);
</span><ins>+ var testEdit = this._editElement(testElement);
</ins><span class="cx"> var test = testCheckbox.test;
</span><span class="cx">
</span><del>- var str = localStorage.getItem(this._localStorageNameForTest(suite, test));
</del><ins>+ var str = localStorage.getItem(this._localStorageNameForTest(suite.name, test.name));
</ins><span class="cx"> if (str === null)
</span><span class="cx"> return;
</span><span class="cx">
</span><span class="cx"> var value = JSON.parse(str);
</span><span class="cx"> testCheckbox.checked = value.checked;
</span><ins>+ testEdit.value = value.complexity;
</ins><span class="cx"> }, this);
</span><span class="cx">
</span><span class="cx"> this._updateSuiteCheckboxState(suiteCheckbox);
</span><span class="lines">@@ -334,13 +367,16 @@
</span><span class="cx"> var tests = [];
</span><span class="cx"> suiteCheckbox.testsElements.forEach(function(testElement) {
</span><span class="cx"> var testCheckbox = this._checkboxElement(testElement);
</span><ins>+ var testEdit = this._editElement(testElement);
</ins><span class="cx"> var test = testCheckbox.test;
</span><span class="cx">
</span><del>- if (testCheckbox.checked)
</del><ins>+ if (testCheckbox.checked) {
+ test.complexity = testEdit.value;
</ins><span class="cx"> tests.push(test);
</span><ins>+ }
</ins><span class="cx">
</span><del>- var value = { checked: testCheckbox.checked };
- localStorage.setItem(this._localStorageNameForTest(suite, test), JSON.stringify(value));
</del><ins>+ var value = { checked: testCheckbox.checked, complexity: testEdit.value };
+ localStorage.setItem(this._localStorageNameForTest(suite.name, test.name), JSON.stringify(value));
</ins><span class="cx"> }, this);
</span><span class="cx">
</span><span class="cx"> if (tests.length)
</span><span class="lines">@@ -348,6 +384,22 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> return suites;
</span><ins>+ },
+
+ updateLocalStorageFromJSON: function(iterationResults)
+ {
+ for (var suiteName in iterationResults[Strings["JSON_RESULTS"][1]]) {
+ var suiteResults = iterationResults[Strings["JSON_RESULTS"][1]][suiteName];
+
+ for (var testName in suiteResults[Strings["JSON_RESULTS"][2]]) {
+ var testResults = suiteResults[Strings["JSON_RESULTS"][2]][testName];
+ var data = testResults[Strings["JSON_EXPERIMENTS"][0]]
+ var complexity = Math.round(data[Strings["JSON_MEASUREMENTS"][0]]);
+
+ var value = { checked: true, complexity: complexity };
+ localStorage.setItem(this._localStorageNameForTest(suiteName, testName), JSON.stringify(value));
+ }
+ }
</ins><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -359,8 +411,14 @@
</span><span class="cx"> optionsManager.updateUIFromLocalStorage();
</span><span class="cx"> suitesManager.createElements();
</span><span class="cx"> suitesManager.updateUIFromLocalStorage();
</span><ins>+ suitesManager.updateEditsElementsState();
</ins><span class="cx"> },
</span><span class="cx">
</span><ins>+ onChangeAdaptiveTestCheckbox: function()
+ {
+ suitesManager.updateEditsElementsState();
+ },
+
</ins><span class="cx"> _runBenchmark: function(suites, options)
</span><span class="cx"> {
</span><span class="cx"> benchmarkRunnerClient.initialize(suites, options);
</span></span></pre></div>
<a id="trunkPerformanceTestsAnimometertestsresourcesmainjs"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/Animometer/tests/resources/main.js (191872 => 191873)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/Animometer/tests/resources/main.js 2015-11-02 01:01:28 UTC (rev 191872)
+++ trunk/PerformanceTests/Animometer/tests/resources/main.js 2015-11-02 02:31:02 UTC (rev 191873)
</span><span class="lines">@@ -179,7 +179,11 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> var tuneValue = 0;
</span><del>- if (!(this._isSampling && this.options["fix-test-complexity"])) {
</del><ins>+ if (this.options["complexity"] && !this.options["adaptive-test"]) {
+ // this.tune(0) returns the current complexity of the test.
+ tuneValue = this.options["complexity"] - this.tune(0);
+ }
+ else if (!(this._isSampling && this.options["fix-test-complexity"])) {
</ins><span class="cx"> // The relationship between frameRate and test complexity is inverse-proportional so we
</span><span class="cx"> // need to use the negative of PIDController.tune() to change the complexity of the test.
</span><span class="cx"> tuneValue = -this._controller.tune(currentFrameRate, timeDelta / 1000);
</span><span class="lines">@@ -237,7 +241,9 @@
</span><span class="cx"> // This function is called from the suite controller run-callback when running the benchmark runner.
</span><span class="cx"> window.runBenchmark = function(suite, test, options, recordTable, progressBar)
</span><span class="cx"> {
</span><del>- var mergedOptions = Utilities.mergeObjects(options, Utilities.parseParameters());
- window.benchmark = window.benchmarkClient.create(suite, test, mergedOptions, recordTable, progressBar);
</del><ins>+ var benchmarkOptions = { complexity: test.complexity };
+ benchmarkOptions = Utilities.mergeObjects(benchmarkOptions, options);
+ benchmarkOptions = Utilities.mergeObjects(benchmarkOptions, Utilities.parseParameters());
+ window.benchmark = window.benchmarkClient.create(suite, test, benchmarkOptions, recordTable, progressBar);
</ins><span class="cx"> return window.benchmark.run();
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkPerformanceTestsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/ChangeLog (191872 => 191873)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/ChangeLog 2015-11-02 01:01:28 UTC (rev 191872)
+++ trunk/PerformanceTests/ChangeLog 2015-11-02 02:31:02 UTC (rev 191873)
</span><span class="lines">@@ -1,5 +1,46 @@
</span><span class="cx"> 2015-11-01 Said Abou-Hallawa <sabouhallawa@apple,com>
</span><span class="cx">
</span><ins>+ Add an option to make the graphics benchmark runs a specific test with fixed complexity
+ https://bugs.webkit.org/show_bug.cgi?id=150529
+
+ Reviewed by Darin Adler.
+
+ Beside each test in the suites tree, we are going to show the complexity
+ arithmetic mean of the of the last run in an edit control. Based on a
+ new option these edit controls will all be visible or hidden. If they are
+ visible their values can be changed. The benchmark runner if it run in
+ the non-adaptive mode will set the complexity of the test to the passed
+ value and will not change it ever. The animator will animate the test and
+ frame rate will also be measured.
+
+ * Animometer/runner/animometer.html: Add a new option for the non-adaptive mode.
+
+ * Animometer/runner/resources/animometer.css:
+ (section#home input[type="number"]): Define the width of all the edit control in the <home> section.
+ (section#home > suites input[type="number"]): The edit controls in the <suites> box will be right aligned and hidden by default.
+ (section#home > suites input[type="number"].selected): When the class "selected" is added, the edit controls will be visible.
+ (section#home > options > label > input[type="number"]): Deleted.
+
+ * Animometer/runner/resources/animometer.js:
+ (window.benchmarkRunnerClient.didFinishLastIteration): Update the local storage with the results of each test.
+ (window.optionsManager._adaptiveTestElement): Returns the checkbox for setting the adaptive test option.
+ (window.suitesManager._editElement): Returns the edit element associated with each test element in the suites tree.
+ (window.suitesManager._editsElements): Returns a list of all the elements in the <suites> box.
+ (window.suitesManager._localStorageNameForTest): Change this function to take strings.
+ (window.suitesManager._createTestElement): Adds an edit control beside each test.
+ (window.suitesManager.updateEditsElementsState): Adds/Removes the 'selected' class to/from all the tests edit elements.
+ (window.suitesManager.updateUIFromLocalStorage): Reads the edit control value from the local storage.
+ (window.suitesManager.updateLocalStorageFromUI): Saves the edit control value to the local storage.
+ (window.suitesManager.updateLocalStorageFromJSON): Saves the last run results to the local storage.
+ (window.benchmarkController.initialize): Shows/Hides the test edit controls based on the adaptive test option.
+ (window.benchmarkController.onChangeAdaptiveTestCheckbox): An onchange event handler for the adaptive test checkbox.
+
+ * Animometer/tests/resources/main.js:
+ (Benchmark.prototype.update): Fix the complexity of the test if the running mode is non-adaptive test and desired complexity is not zero.
+ (window.runBenchmark): Add the test complexity as a new benchmark option.
+
+2015-11-01 Said Abou-Hallawa <sabouhallawa@apple,com>
+
</ins><span class="cx"> Make the size of the benchmark canvas adaptive to the screen size and screen resolution
</span><span class="cx"> https://bugs.webkit.org/show_bug.cgi?id=150530
</span><span class="cx">
</span></span></pre>
</div>
</div>
</body>
</html>