<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[197498] trunk/PerformanceTests</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/197498">197498</a></dd>
<dt>Author</dt> <dd>jonlee@apple.com</dd>
<dt>Date</dt> <dd>2016-03-03 00:15:35 -0800 (Thu, 03 Mar 2016)</dd>
</dl>
<h3>Log Message</h3>
<pre>Add some new controllers, and refine tests
https://bugs.webkit.org/show_bug.cgi?id=154914
Reviewed by Simon Fraser.
Add a controller that centers around 30 fps instead of 60 fps.
* Animometer/developer.html: Add a new option.
* Animometer/resources/debug-runner/animometer.js:
* Animometer/resources/runner/animometer.js:
(this._processData.findRegression): When calculating the complexity-frameLength regression,
check the controller, and use a 30 fps baseline if needed.
* Animometer/resources/statistics.js:
(Regression.Utilities.createClass): Update to allow clients to specify the baseline
frame length.
* Animometer/tests/resources/main.js:
(tune): Override some of the constants in RampController. Move those constants out for
easier reading.
Add a fixed controller, with no step.
* Animometer/developer.html: Add a controller that takes no step.
* Animometer/resources/debug-runner/animometer.js:
* Animometer/tests/resources/main.js:
(Rotater.Utilities.createClass):
Switch to ramp controller as default.
* Animometer/developer.html: Increase the test length to 20 seconds.
* Animometer/resources/debug-runner/animometer.js: Rename the "adjustment" field to "controller"
since that is a more accurate description.
* Animometer/resources/debug-runner/graph.js:
* Animometer/resources/runner/animometer.js: Update preferences for release suite.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkPerformanceTestsAnimometerdeveloperhtml">trunk/PerformanceTests/Animometer/developer.html</a></li>
<li><a href="#trunkPerformanceTestsAnimometerresourcesdebugrunneranimometerjs">trunk/PerformanceTests/Animometer/resources/debug-runner/animometer.js</a></li>
<li><a href="#trunkPerformanceTestsAnimometerresourcesdebugrunnergraphjs">trunk/PerformanceTests/Animometer/resources/debug-runner/graph.js</a></li>
<li><a href="#trunkPerformanceTestsAnimometerresourcesrunneranimometerjs">trunk/PerformanceTests/Animometer/resources/runner/animometer.js</a></li>
<li><a href="#trunkPerformanceTestsAnimometerresourcesstatisticsjs">trunk/PerformanceTests/Animometer/resources/statistics.js</a></li>
<li><a href="#trunkPerformanceTestsAnimometertestsresourcesmainjs">trunk/PerformanceTests/Animometer/tests/resources/main.js</a></li>
<li><a href="#trunkPerformanceTestsChangeLog">trunk/PerformanceTests/ChangeLog</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkPerformanceTestsAnimometerdeveloperhtml"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/Animometer/developer.html (197497 => 197498)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/Animometer/developer.html 2016-03-03 07:44:00 UTC (rev 197497)
+++ trunk/PerformanceTests/Animometer/developer.html 2016-03-03 08:15:35 UTC (rev 197498)
</span><span class="lines">@@ -35,7 +35,7 @@
</span><span class="cx"> <form name="benchmark-options">
</span><span class="cx"> <ul>
</span><span class="cx"> <li>
</span><del>- <label>Test length: <input type="number" id="test-interval" value="10"> seconds each</label>
</del><ins>+ <label>Test length: <input type="number" id="test-interval" value="20"> seconds each</label>
</ins><span class="cx"> </li>
</span><span class="cx"> <li>
</span><span class="cx"> <h3>Display:</h3>
</span><span class="lines">@@ -47,9 +47,11 @@
</span><span class="cx"> <li>
</span><span class="cx"> <h3>Adjusting the test complexity:</h3>
</span><span class="cx"> <ul>
</span><del>- <li><label><input name="adjustment" type="radio" value="step"> Keep at a fixed complexity, then make a big step</label></li>
- <li><label><input name="adjustment" type="radio" value="adaptive" checked> Maintain target FPS</label></li>
- <li><label><input name="adjustment" type="radio" value="ramp"> Ramp</label></li>
</del><ins>+ <li><label><input name="controller" type="radio" value="fixed"> Keep at a fixed complexity</label></li>
+ <li><label><input name="controller" type="radio" value="step"> Keep at a fixed complexity, then make a big step</label></li>
+ <li><label><input name="controller" type="radio" value="adaptive"> Maintain target FPS</label></li>
+ <li><label><input name="controller" type="radio" value="ramp" checked> Ramp</label></li>
+ <li><label><input name="controller" type="radio" value="ramp30"> Ramp @ 30fps</label></li>
</ins><span class="cx"> </ul>
</span><span class="cx"> </li>
</span><span class="cx"> <li>
</span></span></pre></div>
<a id="trunkPerformanceTestsAnimometerresourcesdebugrunneranimometerjs"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/Animometer/resources/debug-runner/animometer.js (197497 => 197498)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/Animometer/resources/debug-runner/animometer.js 2016-03-03 07:44:00 UTC (rev 197497)
+++ trunk/PerformanceTests/Animometer/resources/debug-runner/animometer.js 2016-03-03 08:15:35 UTC (rev 197498)
</span><span class="lines">@@ -328,7 +328,7 @@
</span><span class="cx"> updateEditsElementsState: function()
</span><span class="cx"> {
</span><span class="cx"> var editsElements = this._editsElements();
</span><del>- var showComplexityInputs = optionsManager.valueForOption("adjustment") == "step";
</del><ins>+ var showComplexityInputs = ["fixed", "step"].indexOf(optionsManager.valueForOption("controller")) != -1;
</ins><span class="cx">
</span><span class="cx"> for (var i = 0; i < editsElements.length; ++i) {
</span><span class="cx"> var editElement = editsElements[i];
</span><span class="lines">@@ -471,7 +471,7 @@
</span><span class="cx">
</span><span class="cx"> onBenchmarkOptionsChanged: function(event)
</span><span class="cx"> {
</span><del>- if (event.target.name == "adjustment") {
</del><ins>+ if (event.target.name == "controller") {
</ins><span class="cx"> suitesManager.updateEditsElementsState();
</span><span class="cx"> return;
</span><span class="cx"> }
</span><span class="lines">@@ -495,7 +495,7 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> var dashboard = benchmarkRunnerClient.results;
</span><del>- if (dashboard.options["adjustment"] == "ramp")
</del><ins>+ if (["ramp", "ramp30"].indexOf(dashboard.options["controller"]) != -1)
</ins><span class="cx"> Headers.details[3].disabled = true;
</span><span class="cx"> else {
</span><span class="cx"> Headers.details[1].disabled = true;
</span></span></pre></div>
<a id="trunkPerformanceTestsAnimometerresourcesdebugrunnergraphjs"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/Animometer/resources/debug-runner/graph.js (197497 => 197498)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/Animometer/resources/debug-runner/graph.js 2016-03-03 07:44:00 UTC (rev 197497)
+++ trunk/PerformanceTests/Animometer/resources/debug-runner/graph.js 2016-03-03 08:15:35 UTC (rev 197498)
</span><span class="lines">@@ -368,7 +368,7 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> // right-target
</span><del>- if (options["adjustment"] == "adaptive") {
</del><ins>+ if (options["controller"] == "adaptive") {
</ins><span class="cx"> var targetFrameLength = 1000 / options["frame-rate"];
</span><span class="cx"> svg.append("line")
</span><span class="cx"> .attr("x1", x(0))
</span></span></pre></div>
<a id="trunkPerformanceTestsAnimometerresourcesrunneranimometerjs"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/Animometer/resources/runner/animometer.js (197497 => 197498)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/Animometer/resources/runner/animometer.js 2016-03-03 07:44:00 UTC (rev 197497)
+++ trunk/PerformanceTests/Animometer/resources/runner/animometer.js 2016-03-03 08:15:35 UTC (rev 197498)
</span><span class="lines">@@ -60,6 +60,10 @@
</span><span class="cx"> data[Strings.json.result] = result;
</span><span class="cx"> var samples = data[Strings.json.samples];
</span><span class="cx">
</span><ins>+ var desiredFrameLength = 1000/60;
+ if (this._options["controller"] == "ramp30")
+ desiredFrameLength = 1000/30;
+
</ins><span class="cx"> function findRegression(series) {
</span><span class="cx"> var minIndex = Math.round(.025 * series.length);
</span><span class="cx"> var maxIndex = Math.round(.975 * (series.length - 1));
</span><span class="lines">@@ -80,7 +84,7 @@
</span><span class="cx"> series,
</span><span class="cx"> function (datum, i) { return datum[i].complexity; },
</span><span class="cx"> function (datum, i) { return datum[i].frameLength; },
</span><del>- minIndex, maxIndex)
</del><ins>+ minIndex, maxIndex, desiredFrameLength)
</ins><span class="cx"> };
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -107,7 +111,7 @@
</span><span class="cx"> regression[Strings.json.measurements.stdev] = Math.sqrt(calculation.error / samples[seriesName].length);
</span><span class="cx"> });
</span><span class="cx">
</span><del>- if (this._options["adjustment"] == "ramp") {
</del><ins>+ if (["ramp", "ramp30"].indexOf(this._options["controller"]) != -1) {
</ins><span class="cx"> var timeComplexity = new Experiment;
</span><span class="cx"> data[Strings.json.controller].forEach(function(regression) {
</span><span class="cx"> timeComplexity.sample(regression[Strings.json.complexity]);
</span><span class="lines">@@ -364,10 +368,9 @@
</span><span class="cx"> startBenchmark: function()
</span><span class="cx"> {
</span><span class="cx"> var options = {
</span><del>- "test-interval": 10,
</del><ins>+ "test-interval": 20,
</ins><span class="cx"> "display": "minimal",
</span><del>- "adjustment": "adaptive",
- "frame-rate": 50,
</del><ins>+ "controller": "ramp",
</ins><span class="cx"> "kalman-process-error": 1,
</span><span class="cx"> "kalman-measurement-error": 4,
</span><span class="cx"> "time-measurement": "performance"
</span></span></pre></div>
<a id="trunkPerformanceTestsAnimometerresourcesstatisticsjs"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/Animometer/resources/statistics.js (197497 => 197498)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/Animometer/resources/statistics.js 2016-03-03 07:44:00 UTC (rev 197497)
+++ trunk/PerformanceTests/Animometer/resources/statistics.js 2016-03-03 08:15:35 UTC (rev 197498)
</span><span class="lines">@@ -148,16 +148,17 @@
</span><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> Regression = Utilities.createClass(
</span><del>- function(samples, getComplexity, getFrameLength, startIndex, endIndex, options)
</del><ins>+ function(samples, getComplexity, getFrameLength, startIndex, endIndex, desiredFrameLength)
</ins><span class="cx"> {
</span><ins>+ desiredFrameLength = desiredFrameLength || 1000/60;
</ins><span class="cx"> var slope = this._calculateRegression(samples, getComplexity, getFrameLength, startIndex, endIndex, {
</span><span class="cx"> shouldClip: true,
</span><del>- s1: 1000/60,
</del><ins>+ s1: desiredFrameLength,
</ins><span class="cx"> t1: 0
</span><span class="cx"> });
</span><span class="cx"> var flat = this._calculateRegression(samples, getComplexity, getFrameLength, startIndex, endIndex, {
</span><span class="cx"> shouldClip: true,
</span><del>- s1: 1000/60,
</del><ins>+ s1: desiredFrameLength,
</ins><span class="cx"> t1: 0,
</span><span class="cx"> t2: 0
</span><span class="cx"> });
</span></span></pre></div>
<a id="trunkPerformanceTestsAnimometertestsresourcesmainjs"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/Animometer/tests/resources/main.js (197497 => 197498)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/Animometer/tests/resources/main.js 2016-03-03 07:44:00 UTC (rev 197497)
+++ trunk/PerformanceTests/Animometer/tests/resources/main.js 2016-03-03 08:15:35 UTC (rev 197498)
</span><span class="lines">@@ -51,7 +51,7 @@
</span><span class="cx"> this._isFrameLengthEstimatorEnabled = true;
</span><span class="cx">
</span><span class="cx"> // Length of subsequent intervals; a value of 0 means use no intervals
</span><del>- this._intervalLength = options["interval-length"] || 100;
</del><ins>+ this.intervalSamplingLength = 100;
</ins><span class="cx">
</span><span class="cx"> this.initialComplexity = 0;
</span><span class="cx"> }, {
</span><span class="lines">@@ -97,7 +97,7 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> this._intervalStartIndex = sampleCount;
</span><del>- this._intervalEndTimestamp = currentTimestamp + this._intervalLength;
</del><ins>+ this._intervalEndTimestamp = currentTimestamp + this.intervalSamplingLength;
</ins><span class="cx">
</span><span class="cx"> return averageFrameLength;
</span><span class="cx"> },
</span><span class="lines">@@ -109,7 +109,7 @@
</span><span class="cx">
</span><span class="cx"> var frameLengthEstimate = -1, intervalAverageFrameLength = -1;
</span><span class="cx"> var didFinishInterval = false;
</span><del>- if (!this._intervalLength) {
</del><ins>+ if (!this.intervalSamplingLength) {
</ins><span class="cx"> if (this._isFrameLengthEstimatorEnabled) {
</span><span class="cx"> this._frameLengthEstimator.sample(lastFrameLength);
</span><span class="cx"> frameLengthEstimate = this._frameLengthEstimator.estimate;
</span><span class="lines">@@ -219,12 +219,21 @@
</span><span class="cx"> }
</span><span class="cx"> });
</span><span class="cx">
</span><ins>+FixedController = Utilities.createSubclass(Controller,
+ function(benchmark, options)
+ {
+ Controller.call(this, benchmark, options);
+ this.initialComplexity = options["complexity"];
+ this.intervalSamplingLength = 0;
+ }
+);
+
</ins><span class="cx"> StepController = Utilities.createSubclass(Controller,
</span><span class="cx"> function(benchmark, options)
</span><span class="cx"> {
</span><del>- options["interval-length"] = 0;
</del><span class="cx"> Controller.call(this, benchmark, options);
</span><span class="cx"> this.initialComplexity = options["complexity"];
</span><ins>+ this.intervalSamplingLength = 0;
</ins><span class="cx"> this._stepped = false;
</span><span class="cx"> this._stepTime = options["test-interval"] / 2;
</span><span class="cx"> }, {
</span><span class="lines">@@ -318,12 +327,6 @@
</span><span class="cx"> this._tier = -.5;
</span><span class="cx"> // The timestamp is first set after the first interval completes
</span><span class="cx"> this._tierStartTimestamp = 0;
</span><del>- // If the engine can handle the tier's complexity at 60 FPS, test for a short
- // period, then move on to the next tier
- this._tierFastTestLength = 250;
- // If the engine is under stress, let the test run a little longer to let
- // the measurement settle
- this._tierSlowTestLength = 750;
</del><span class="cx"> this._minimumComplexity = 0;
</span><span class="cx"> this._maximumComplexity = 0;
</span><span class="cx">
</span><span class="lines">@@ -331,18 +334,10 @@
</span><span class="cx"> var minimumRampLength = 2500;
</span><span class="cx"> var totalRampIterations = Math.max(1, Math.floor(this._endTimestamp / minimumRampLength));
</span><span class="cx"> // Give a little extra room to run since the ramps won't be exactly this length
</span><del>- this._rampLength = Math.floor((this._endTimestamp - totalRampIterations * this._intervalLength) / totalRampIterations);
- this._rampWarmupLength = 200;
</del><ins>+ this._rampLength = Math.floor((this._endTimestamp - totalRampIterations * this.intervalSamplingLength) / totalRampIterations);
</ins><span class="cx"> this._rampDidWarmup = false;
</span><span class="cx"> this._rampRegressions = [];
</span><span class="cx">
</span><del>- // Add some tolerance; frame lengths shorter than this are considered to be @ 60 fps
- this._fps60Threshold = 1000/58;
- // We are looking for the complexity that will get us at least as slow this threshold
- this._fpsLowestThreshold = 1000/30;
- // Try to make each ramp get this slow so that we can cross the break point
- this._fpsRampSlowThreshold = 1000/45;
-
</del><span class="cx"> this._finishedTierSampling = false;
</span><span class="cx"> this._changePointEstimator = new Experiment;
</span><span class="cx"> this._minimumComplexityEstimator = new Experiment;
</span><span class="lines">@@ -350,6 +345,23 @@
</span><span class="cx"> this._intervalFrameLengthEstimator = new Experiment;
</span><span class="cx"> }, {
</span><span class="cx">
</span><ins>+ // If the engine can handle the tier's complexity at the desired frame rate, test for a short
+ // period, then move on to the next tier
+ tierFastTestLength: 250,
+ // If the engine is under stress, let the test run a little longer to let the measurement settle
+ tierSlowTestLength: 750,
+
+ rampWarmupLength: 200,
+
+ // Used for regression calculations in the ramps
+ frameLengthDesired: 1000/60,
+ // Add some tolerance; frame lengths shorter than this are considered to be @ the desired frame length
+ frameLengthDesiredThreshold: 1000/58,
+ // Represents the lower bound threshold in order to find the right complexity range to sample
+ frameLengthSlowestThreshold: 1000/30,
+ // Try to make each ramp get this slow so that we can cross the break point
+ frameLengthRampLowerThreshold: 1000/45,
+
</ins><span class="cx"> start: function(startTimestamp, stage)
</span><span class="cx"> {
</span><span class="cx"> Controller.prototype.start.call(this, startTimestamp, stage);
</span><span class="lines">@@ -359,14 +371,14 @@
</span><span class="cx"> didFinishInterval: function(timestamp, stage, intervalAverageFrameLength)
</span><span class="cx"> {
</span><span class="cx"> if (!this._finishedTierSampling) {
</span><del>- if (this._tierStartTimestamp > 0 && timestamp < this._tierStartTimestamp + this._tierFastTestLength)
</del><ins>+ if (this._tierStartTimestamp > 0 && timestamp < this._tierStartTimestamp + this.tierFastTestLength)
</ins><span class="cx"> return;
</span><span class="cx">
</span><span class="cx"> var currentComplexity = stage.complexity();
</span><span class="cx"> var currentFrameLength = this._frameLengthEstimator.estimate;
</span><del>- if (currentFrameLength < this._fpsLowestThreshold) {
- var isAnimatingAt60FPS = currentFrameLength < this._fps60Threshold;
- var hasFinishedSlowTierTest = timestamp > this._tierStartTimestamp + this._tierSlowTestLength;
</del><ins>+ if (currentFrameLength < this.frameLengthSlowestThreshold) {
+ var isAnimatingAt60FPS = currentFrameLength < this.frameLengthDesiredThreshold;
+ var hasFinishedSlowTierTest = timestamp > this._tierStartTimestamp + this.tierSlowTestLength;
</ins><span class="cx">
</span><span class="cx"> if (!isAnimatingAt60FPS && !hasFinishedSlowTierTest)
</span><span class="cx"> return;
</span><span class="lines">@@ -387,7 +399,7 @@
</span><span class="cx"> this.mark("Complexity: " + nextTierComplexity, timestamp);
</span><span class="cx"> return;
</span><span class="cx"> }
</span><del>- } else if (timestamp < this._tierStartTimestamp + this._tierSlowTestLength)
</del><ins>+ } else if (timestamp < this._tierStartTimestamp + this.tierSlowTestLength)
</ins><span class="cx"> return;
</span><span class="cx">
</span><span class="cx"> this._finishedTierSampling = true;
</span><span class="lines">@@ -405,7 +417,7 @@
</span><span class="cx"> // Avoid going down that far since it means fewer measurements are taken in the 60 fps area
</span><span class="cx"> // Interpolate a maximum complexity that gets us around the lowest threshold
</span><span class="cx"> if (this._lastTierComplexity != currentComplexity)
</span><del>- this._maximumComplexity = Math.floor(Utilities.lerp(Utilities.progressValue(this._fpsLowestThreshold, this._lastTierFrameLength, currentFrameLength), this._lastTierComplexity, currentComplexity));
</del><ins>+ this._maximumComplexity = Math.floor(Utilities.lerp(Utilities.progressValue(this.frameLengthSlowestThreshold, this._lastTierFrameLength, currentFrameLength), this._lastTierComplexity, currentComplexity));
</ins><span class="cx"> else {
</span><span class="cx"> // If the browser is capable of handling the most complex version of the test, use that
</span><span class="cx"> this._maximumComplexity = currentComplexity;
</span><span class="lines">@@ -425,7 +437,7 @@
</span><span class="cx"> return;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- if ((timestamp - this._rampStartTimestamp) < this._rampWarmupLength)
</del><ins>+ if ((timestamp - this._rampStartTimestamp) < this.rampWarmupLength)
</ins><span class="cx"> return;
</span><span class="cx">
</span><span class="cx"> if (this._rampDidWarmup)
</span><span class="lines">@@ -451,14 +463,14 @@
</span><span class="cx"> var intervalFrameLengthMean = this._intervalFrameLengthEstimator.mean();
</span><span class="cx"> var intervalFrameLengthStandardDeviation = this._intervalFrameLengthEstimator.standardDeviation();
</span><span class="cx">
</span><del>- if (intervalFrameLengthMean < this._fps60Threshold && this._intervalFrameLengthEstimator.cdf(this._fps60Threshold) > .95) {
</del><ins>+ if (intervalFrameLengthMean < this.frameLengthDesiredThreshold && this._intervalFrameLengthEstimator.cdf(this.frameLengthDesiredThreshold) > .95) {
</ins><span class="cx"> this._possibleMinimumComplexity = Math.max(this._possibleMinimumComplexity, currentComplexity);
</span><span class="cx"> } else if (intervalFrameLengthStandardDeviation > 2) {
</span><span class="cx"> // In the case where we might have found a previous interval where 60fps was reached. We hit a significant blip,
</span><span class="cx"> // so we should resample this area in the next ramp.
</span><span class="cx"> this._possibleMinimumComplexity = 0;
</span><span class="cx"> }
</span><del>- if (intervalFrameLengthMean - intervalFrameLengthStandardDeviation > this._fpsRampSlowThreshold)
</del><ins>+ if (intervalFrameLengthMean - intervalFrameLengthStandardDeviation > this.frameLengthRampLowerThreshold)
</ins><span class="cx"> this._possibleMaximumComplexity = Math.min(this._possibleMaximumComplexity, currentComplexity);
</span><span class="cx"> this._intervalFrameLengthEstimator.reset();
</span><span class="cx">
</span><span class="lines">@@ -470,12 +482,12 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> var regression = new Regression(this._sampler.samples, this._getComplexity, this._getFrameLength,
</span><del>- this._sampler.sampleCount - 1, this._rampStartIndex);
</del><ins>+ this._sampler.sampleCount - 1, this._rampStartIndex, this.frameLengthDesired);
</ins><span class="cx"> this._rampRegressions.push(regression);
</span><span class="cx">
</span><span class="cx"> var interpolatedFrameLength = regression.valueAt(this._maximumComplexity);
</span><del>- if (interpolatedFrameLength < this._fpsRampSlowThreshold)
- this._possibleMaximumComplexity = Math.floor(Utilities.lerp(Utilities.progressValue(this._fpsRampSlowThreshold, interpolatedFrameLength, this._lastTierFrameLength), this._maximumComplexity, this._lastTierComplexity));
</del><ins>+ if (interpolatedFrameLength < this.frameLengthRampLowerThreshold)
+ this._possibleMaximumComplexity = Math.floor(Utilities.lerp(Utilities.progressValue(this.frameLengthRampLowerThreshold, interpolatedFrameLength, this._lastTierFrameLength), this._maximumComplexity, this._lastTierComplexity));
</ins><span class="cx">
</span><span class="cx"> interpolatedFrameLength = regression.valueAt(this._minimumComplexity);
</span><span class="cx"> this._minimumComplexityEstimator.sample(this._possibleMinimumComplexity);
</span><span class="lines">@@ -557,6 +569,18 @@
</span><span class="cx"> }
</span><span class="cx"> });
</span><span class="cx">
</span><ins>+Ramp30Controller = Utilities.createSubclass(RampController,
+ function(benchmark, options)
+ {
+ RampController.call(this, benchmark, options);
+ }, {
+
+ frameLengthDesired: 1000/30,
+ frameLengthDesiredThreshold: 1000/29,
+ frameLengthSlowestThreshold: 1000/20,
+ frameLengthRampLowerThreshold: 1000/20
+});
+
</ins><span class="cx"> Stage = Utilities.createClass(
</span><span class="cx"> function()
</span><span class="cx"> {
</span><span class="lines">@@ -674,7 +698,7 @@
</span><span class="cx"> 'color',
</span><span class="cx"> 'luminosity'
</span><span class="cx"> ];
</span><del>-
</del><ins>+
</ins><span class="cx"> return mixBlendModeList[this.randomInt(0, mixBlendModeList.length)];
</span><span class="cx"> },
</span><span class="cx">
</span><span class="lines">@@ -692,7 +716,7 @@
</span><span class="cx"> 'blur(10px)',
</span><span class="cx"> 'drop-shadow(10px 10px 10px gray)'
</span><span class="cx"> ];
</span><del>-
</del><ins>+
</ins><span class="cx"> return filterList[this.randomInt(0, filterList.length)];
</span><span class="cx"> },
</span><span class="cx">
</span><span class="lines">@@ -775,8 +799,11 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> options["test-interval"] *= 1000;
</span><del>- switch (options["adjustment"])
</del><ins>+ switch (options["controller"])
</ins><span class="cx"> {
</span><ins>+ case "fixed":
+ this._controller = new FixedController(this, options);
+ break;
</ins><span class="cx"> case "step":
</span><span class="cx"> this._controller = new StepController(this, options);
</span><span class="cx"> break;
</span><span class="lines">@@ -786,6 +813,8 @@
</span><span class="cx"> case "ramp":
</span><span class="cx"> this._controller = new RampController(this, options);
</span><span class="cx"> break;
</span><ins>+ case "ramp30":
+ this._controller = new Ramp30Controller(this, options);
</ins><span class="cx"> }
</span><span class="cx"> }, {
</span><span class="cx">
</span></span></pre></div>
<a id="trunkPerformanceTestsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/ChangeLog (197497 => 197498)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/ChangeLog 2016-03-03 07:44:00 UTC (rev 197497)
+++ trunk/PerformanceTests/ChangeLog 2016-03-03 08:15:35 UTC (rev 197498)
</span><span class="lines">@@ -1,3 +1,39 @@
</span><ins>+2016-03-02 Jon Lee <jonlee@apple.com>
+
+ Add some new controllers, and refine tests
+ https://bugs.webkit.org/show_bug.cgi?id=154914
+
+ Reviewed by Simon Fraser.
+
+ Add a controller that centers around 30 fps instead of 60 fps.
+
+ * Animometer/developer.html: Add a new option.
+ * Animometer/resources/debug-runner/animometer.js:
+ * Animometer/resources/runner/animometer.js:
+ (this._processData.findRegression): When calculating the complexity-frameLength regression,
+ check the controller, and use a 30 fps baseline if needed.
+ * Animometer/resources/statistics.js:
+ (Regression.Utilities.createClass): Update to allow clients to specify the baseline
+ frame length.
+ * Animometer/tests/resources/main.js:
+ (tune): Override some of the constants in RampController. Move those constants out for
+ easier reading.
+
+ Add a fixed controller, with no step.
+
+ * Animometer/developer.html: Add a controller that takes no step.
+ * Animometer/resources/debug-runner/animometer.js:
+ * Animometer/tests/resources/main.js:
+ (Rotater.Utilities.createClass):
+
+ Switch to ramp controller as default.
+
+ * Animometer/developer.html: Increase the test length to 20 seconds.
+ * Animometer/resources/debug-runner/animometer.js: Rename the "adjustment" field to "controller"
+ since that is a more accurate description.
+ * Animometer/resources/debug-runner/graph.js:
+ * Animometer/resources/runner/animometer.js: Update preferences for release suite.
+
</ins><span class="cx"> 2016-02-26 Jon Lee <jonlee@apple.com>
</span><span class="cx">
</span><span class="cx"> Address Dean's comments in 154673.
</span></span></pre>
</div>
</div>
</body>
</html>