<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[213998] trunk/Websites/perf.webkit.org</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/213998">213998</a></dd>
<dt>Author</dt> <dd>rniwa@webkit.org</dd>
<dt>Date</dt> <dd>2017-03-15 12:15:02 -0700 (Wed, 15 Mar 2017)</dd>
</dl>
<h3>Log Message</h3>
<pre>Add tests for privileged-api/create-analysis-task and privileged-api/create-test-group
https://bugs.webkit.org/show_bug.cgi?id=169688
Rubber-stamped by Antti Koivisto.
Added tests for privileged-api/create-analysis-task and privileged-api/create-test-group, and fixed newly found bugs.
* public/privileged-api/create-analysis-task.php:
(main): Fixed the bug that we were not explicitly checking whether start_run and end_run were integers or not.
Also return InvalidTimeRange when start and end times are identical as that makes no sense for an analysis task.
* public/privileged-api/create-test-group.php:
(main): Fixed a bug that we were not explicitly checking task and repetitionCount to be an integer.
(ensure_commit_sets): Fixed the bug that the number of commit sets weren't checked.
* server-tests/privileged-api-create-analysis-task-tests.js: Added.
* server-tests/privileged-api-create-test-group-tests.js: Added.
* server-tests/resources/common-operations.js:
(prepareServerTest): Increase the timeout from 1s to 5s.
* server-tests/resources/mock-data.js:
(MockData.addMockData): Use a higher database ID of 20 for a mock build_slave to avoid a conflict with auto-generated IDs.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkWebsitesperfwebkitorgChangeLog">trunk/Websites/perf.webkit.org/ChangeLog</a></li>
<li><a href="#trunkWebsitesperfwebkitorgpublicprivilegedapicreateanalysistaskphp">trunk/Websites/perf.webkit.org/public/privileged-api/create-analysis-task.php</a></li>
<li><a href="#trunkWebsitesperfwebkitorgpublicprivilegedapicreatetestgroupphp">trunk/Websites/perf.webkit.org/public/privileged-api/create-test-group.php</a></li>
<li><a href="#trunkWebsitesperfwebkitorgservertestsresourcescommonoperationsjs">trunk/Websites/perf.webkit.org/server-tests/resources/common-operations.js</a></li>
<li><a href="#trunkWebsitesperfwebkitorgservertestsresourcesmockdatajs">trunk/Websites/perf.webkit.org/server-tests/resources/mock-data.js</a></li>
</ul>
<h3>Added Paths</h3>
<ul>
<li><a href="#trunkWebsitesperfwebkitorgservertestsprivilegedapicreateanalysistasktestsjs">trunk/Websites/perf.webkit.org/server-tests/privileged-api-create-analysis-task-tests.js</a></li>
<li><a href="#trunkWebsitesperfwebkitorgservertestsprivilegedapicreatetestgrouptestsjs">trunk/Websites/perf.webkit.org/server-tests/privileged-api-create-test-group-tests.js</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkWebsitesperfwebkitorgChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Websites/perf.webkit.org/ChangeLog (213997 => 213998)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/ChangeLog 2017-03-15 19:07:55 UTC (rev 213997)
+++ trunk/Websites/perf.webkit.org/ChangeLog 2017-03-15 19:15:02 UTC (rev 213998)
</span><span class="lines">@@ -1,5 +1,31 @@
</span><span class="cx"> 2017-03-15 Ryosuke Niwa <rniwa@webkit.org>
</span><span class="cx">
</span><ins>+ Add tests for privileged-api/create-analysis-task and privileged-api/create-test-group
+ https://bugs.webkit.org/show_bug.cgi?id=169688
+
+ Rubber-stamped by Antti Koivisto.
+
+ Added tests for privileged-api/create-analysis-task and privileged-api/create-test-group, and fixed newly found bugs.
+
+ * public/privileged-api/create-analysis-task.php:
+ (main): Fixed the bug that we were not explicitly checking whether start_run and end_run were integers or not.
+ Also return InvalidTimeRange when start and end times are identical as that makes no sense for an analysis task.
+
+ * public/privileged-api/create-test-group.php:
+ (main): Fixed a bug that we were not explicitly checking task and repetitionCount to be an integer.
+ (ensure_commit_sets): Fixed the bug that the number of commit sets weren't checked.
+
+ * server-tests/privileged-api-create-analysis-task-tests.js: Added.
+ * server-tests/privileged-api-create-test-group-tests.js: Added.
+
+ * server-tests/resources/common-operations.js:
+ (prepareServerTest): Increase the timeout from 1s to 5s.
+
+ * server-tests/resources/mock-data.js:
+ (MockData.addMockData): Use a higher database ID of 20 for a mock build_slave to avoid a conflict with auto-generated IDs.
+
+2017-03-15 Ryosuke Niwa <rniwa@webkit.org>
+
</ins><span class="cx"> Make unit tests return a promise instead of manually calling done
</span><span class="cx"> https://bugs.webkit.org/show_bug.cgi?id=169663
</span><span class="cx">
</span></span></pre></div>
<a id="trunkWebsitesperfwebkitorgpublicprivilegedapicreateanalysistaskphp"></a>
<div class="modfile"><h4>Modified: trunk/Websites/perf.webkit.org/public/privileged-api/create-analysis-task.php (213997 => 213998)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/public/privileged-api/create-analysis-task.php 2017-03-15 19:07:55 UTC (rev 213997)
+++ trunk/Websites/perf.webkit.org/public/privileged-api/create-analysis-task.php 2017-03-15 19:15:02 UTC (rev 213998)
</span><span class="lines">@@ -8,8 +8,6 @@
</span><span class="cx">
</span><span class="cx"> $author = remote_user_name($data);
</span><span class="cx"> $name = array_get($data, 'name');
</span><del>- $start_run_id = array_get($data, 'startRun');
- $end_run_id = array_get($data, 'endRun');
</del><span class="cx">
</span><span class="cx"> $segmentation_name = array_get($data, 'segmentationStrategy');
</span><span class="cx"> $test_range_name = array_get($data, 'testRangeStrategy');
</span><span class="lines">@@ -16,18 +14,19 @@
</span><span class="cx">
</span><span class="cx"> if (!$name)
</span><span class="cx"> exit_with_error('MissingName', array('name' => $name));
</span><del>- $range = array('startRunId' => $start_run_id, 'endRunId' => $end_run_id);
- if (!$start_run_id || !$end_run_id)
- exit_with_error('MissingRange', $range);
</del><span class="cx">
</span><del>- $start_run = ensure_row_by_id($db, 'test_runs', 'run', $start_run_id, 'InvalidStartRun', $range);
- $end_run = ensure_row_by_id($db, 'test_runs', 'run', $end_run_id, 'InvalidEndRun', $range);
</del><ins>+ $range = validate_arguments($data, array('startRun' => 'int', 'endRun' => 'int'));
</ins><span class="cx">
</span><ins>+ $start_run = ensure_row_by_id($db, 'test_runs', 'run', $range['startRun'], 'InvalidStartRun', $range);
+ $start_run_id = $start_run['run_id'];
+ $end_run = ensure_row_by_id($db, 'test_runs', 'run', $range['endRun'], 'InvalidEndRun', $range);
+ $end_run_id = $end_run['run_id'];
+
</ins><span class="cx"> $config = ensure_config_from_runs($db, $start_run, $end_run);
</span><span class="cx">
</span><span class="cx"> $start_run_time = time_for_run($db, $start_run_id);
</span><span class="cx"> $end_run_time = time_for_run($db, $end_run_id);
</span><del>- if (!$start_run_time || !$end_run_time)
</del><ins>+ if (!$start_run_time || !$end_run_time || $start_run_time == $end_run_time)
</ins><span class="cx"> exit_with_error('InvalidTimeRange', array('startTime' => $start_run_time, 'endTime' => $end_run_time));
</span><span class="cx">
</span><span class="cx"> $db->begin_transaction();
</span></span></pre></div>
<a id="trunkWebsitesperfwebkitorgpublicprivilegedapicreatetestgroupphp"></a>
<div class="modfile"><h4>Modified: trunk/Websites/perf.webkit.org/public/privileged-api/create-test-group.php (213997 => 213998)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/public/privileged-api/create-test-group.php 2017-03-15 19:07:55 UTC (rev 213997)
+++ trunk/Websites/perf.webkit.org/public/privileged-api/create-test-group.php 2017-03-15 19:15:02 UTC (rev 213998)
</span><span class="lines">@@ -7,16 +7,23 @@
</span><span class="cx"> $data = ensure_privileged_api_data_and_token_or_slave($db);
</span><span class="cx"> $author = remote_user_name($data);
</span><span class="cx">
</span><del>- $task_id = array_get($data, 'task');
- $name = array_get($data, 'name');
</del><ins>+ $arguments = validate_arguments($data, array(
+ 'name' => '/.+/',
+ 'task' => 'int',
+ 'repetitionCount' => 'int?',
+ ));
+ $name = $arguments['name'];
+ $task_id = $arguments['task'];
+ $repetition_count = $arguments['repetitionCount'];
</ins><span class="cx"> $commit_sets_info = array_get($data, 'commitSets');
</span><del>- $repetition_count = intval(array_get($data, 'repetitionCount', 1));
</del><span class="cx">
</span><del>- if (!$name)
- exit_with_error('MissingName');
</del><ins>+ require_format('Task', $task_id, '/^\d+$/');
</ins><span class="cx"> if (!$commit_sets_info)
</span><del>- exit_with_error('MissingCommitSets');
- if ($repetition_count < 1)
</del><ins>+ exit_with_error('InvalidCommitSets');
+
+ if ($repetition_count === null)
+ $repetition_count = 1;
+ else if ($repetition_count < 1)
</ins><span class="cx"> exit_with_error('InvalidRepetitionCount', array('repetitionCount' => $repetition_count));
</span><span class="cx">
</span><span class="cx"> $task = $db->select_first_row('analysis_tasks', 'task', array('id' => $task_id));
</span><span class="lines">@@ -80,6 +87,9 @@
</span><span class="cx"> }
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ if (count($commit_sets) < 2)
+ exit_with_error('InvalidCommitSets', array('commitSets' => $commit_sets_info));
+
</ins><span class="cx"> $commit_count_per_set = count($commit_sets[0]);
</span><span class="cx"> foreach ($commit_sets as $commits) {
</span><span class="cx"> if ($commit_count_per_set != count($commits))
</span></span></pre></div>
<a id="trunkWebsitesperfwebkitorgservertestsprivilegedapicreateanalysistasktestsjs"></a>
<div class="addfile"><h4>Added: trunk/Websites/perf.webkit.org/server-tests/privileged-api-create-analysis-task-tests.js (0 => 213998)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/server-tests/privileged-api-create-analysis-task-tests.js (rev 0)
+++ trunk/Websites/perf.webkit.org/server-tests/privileged-api-create-analysis-task-tests.js 2017-03-15 19:15:02 UTC (rev 213998)
</span><span class="lines">@@ -0,0 +1,255 @@
</span><ins>+'use strict';
+
+let assert = require('assert');
+
+let MockData = require('./resources/mock-data.js');
+let TestServer = require('./resources/test-server.js');
+const addBuilderForReport = require('./resources/common-operations.js').addBuilderForReport;
+const prepareServerTest = require('./resources/common-operations.js').prepareServerTest;
+
+const reportWithRevision = [{
+ "buildNumber": "124",
+ "buildTime": "2015-10-27T15:34:51",
+ "revisions": {
+ "WebKit": {
+ "revision": "191622",
+ "timestamp": '2015-10-27T11:36:56.878473Z',
+ },
+ },
+ "builderName": "someBuilder",
+ "builderPassword": "somePassword",
+ "platform": "some platform",
+ "tests": {
+ "Suite": {
+ "metrics": {
+ "Time": ["Arithmetic"],
+ },
+ "tests": {
+ "test1": {
+ "metrics": {"Time": { "current": [11] }},
+ }
+ }
+ },
+ }}];
+
+const anotherReportWithRevision = [{
+ "buildNumber": "125",
+ "buildTime": "2015-10-27T17:27:41",
+ "revisions": {
+ "WebKit": {
+ "revision": "191623",
+ "timestamp": '2015-10-27T16:38:10.768995Z',
+ },
+ },
+ "builderName": "someBuilder",
+ "builderPassword": "somePassword",
+ "platform": "some platform",
+ "tests": {
+ "Suite": {
+ "metrics": {
+ "Time": ["Arithmetic"],
+ },
+ "tests": {
+ "test1": {
+ "metrics": {"Time": { "current": [12] }},
+ }
+ }
+ },
+ }}];
+
+describe('/privileged-api/create-analysis-task', function () {
+ prepareServerTest(this);
+
+ it('should return "MissingName" on an empty request', () => {
+ return PrivilegedAPI.sendRequest('create-analysis-task', {}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'MissingName');
+ });
+ });
+
+ it('should return "InvalidStartRun" when startRun is missing but endRun is set', () => {
+ return PrivilegedAPI.sendRequest('create-analysis-task', {name: 'hi', endRun: 1}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidStartRun');
+ });
+ });
+
+ it('should return "InvalidEndRun" when endRun is missing but startRun is set', () => {
+ return PrivilegedAPI.sendRequest('create-analysis-task', {name: 'hi', startRun: 1}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidEndRun');
+ });
+ });
+
+ it('should return "InvalidStartRun" when startRun is not a valid integer', () => {
+ return PrivilegedAPI.sendRequest('create-analysis-task', {name: 'hi', startRun: "foo", endRun: 1}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidStartRun');
+ });
+ });
+
+ it('should return "InvalidEndRun" when endRun is not a valid integer', () => {
+ return PrivilegedAPI.sendRequest('create-analysis-task', {name: 'hi', startRun: 1, endRun: "foo"}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidEndRun');
+ });
+ });
+
+ it('should return "InvalidStartRun" when startRun is invalid', () => {
+ return addBuilderForReport(reportWithRevision[0]).then(() => {
+ return TestServer.remoteAPI().postJSON('/api/report/', reportWithRevision);
+ }).then(() => {
+ return PrivilegedAPI.sendRequest('create-analysis-task', {name: 'hi', startRun: 100, endRun: 1}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidStartRun');
+ });
+ });
+ });
+
+ it('should return "InvalidEndRun" when endRun is invalid', () => {
+ return addBuilderForReport(reportWithRevision[0]).then(() => {
+ return TestServer.remoteAPI().postJSON('/api/report/', reportWithRevision);
+ }).then(() => {
+ return PrivilegedAPI.sendRequest('create-analysis-task', {name: 'hi', startRun: 1, endRun: 100}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidEndRun');
+ });
+ });
+ });
+
+ it('should return "InvalidTimeRange" when startRun and endRun are identical', () => {
+ return addBuilderForReport(reportWithRevision[0]).then(() => {
+ return TestServer.remoteAPI().postJSON('/api/report/', reportWithRevision);
+ }).then(() => {
+ return PrivilegedAPI.sendRequest('create-analysis-task', {name: 'hi', startRun: 1, endRun: 1}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidTimeRange');
+ });
+ });
+ });
+
+ it('should return "RunConfigMismatch" when startRun and endRun come from a different test configurations', () => {
+ return addBuilderForReport(reportWithRevision[0]).then(() => {
+ return TestServer.remoteAPI().postJSON('/api/report/', reportWithRevision);
+ }).then(() => {
+ return PrivilegedAPI.sendRequest('create-analysis-task', {name: 'hi', startRun: 1, endRun: 2}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'RunConfigMismatch');
+ });
+ });
+ });
+
+ it('should create an analysis task when name, startRun, and endRun are set properly', () => {
+ const db = TestServer.database();
+ return addBuilderForReport(reportWithRevision[0]).then(() => {
+ return TestServer.remoteAPI().postJSON('/api/report/', reportWithRevision);
+ }).then(() => {
+ return TestServer.remoteAPI().postJSON('/api/report/', anotherReportWithRevision);
+ }).then(() => {
+ return Manifest.fetch();
+ }).then(() => {
+ const test1 = Test.findByPath(['Suite', 'test1']);
+ const platform = Platform.findByName('some platform');
+ return db.selectFirstRow('test_configurations', {metric: test1.metrics()[0].id(), platform: platform.id()});
+ }).then((configRow) => {
+ return db.selectRows('test_runs', {config: configRow['id']});
+ }).then((testRuns) => {
+ assert.equal(testRuns.length, 2);
+ return PrivilegedAPI.sendRequest('create-analysis-task', {name: 'hi', startRun: testRuns[0]['id'], endRun: testRuns[1]['id']});
+ }).then((content) => {
+ return AnalysisTask.fetchById(content['taskId']);
+ }).then((task) => {
+ assert.equal(task.name(), 'hi');
+ assert(!task.hasResults());
+ assert(!task.hasPendingRequests());
+ assert.deepEqual(task.bugs(), []);
+ assert.deepEqual(task.causes(), []);
+ assert.deepEqual(task.fixes(), []);
+ assert.equal(task.changeType(), null);
+ assert.equal(task.platform().label(), 'some platform');
+ assert.equal(task.metric().test().label(), 'test1');
+ });
+ });
+
+ it('should return "DuplicateAnalysisTask" when there is already an analysis task for the specified range', () => {
+ const db = TestServer.database();
+ let startId;
+ let endId;
+ return addBuilderForReport(reportWithRevision[0]).then(() => {
+ return TestServer.remoteAPI().postJSON('/api/report/', reportWithRevision);
+ }).then(() => {
+ return TestServer.remoteAPI().postJSON('/api/report/', anotherReportWithRevision);
+ }).then(() => {
+ return Manifest.fetch();
+ }).then(() => {
+ const test1 = Test.findByPath(['Suite', 'test1']);
+ const platform = Platform.findByName('some platform');
+ return db.selectFirstRow('test_configurations', {metric: test1.metrics()[0].id(), platform: platform.id()});
+ }).then((configRow) => {
+ return db.selectRows('test_runs', {config: configRow['id']});
+ }).then((testRuns) => {
+ assert.equal(testRuns.length, 2);
+ startId = testRuns[0]['id'];
+ endId = testRuns[1]['id'];
+ return PrivilegedAPI.sendRequest('create-analysis-task', {name: 'hi', startRun: startId, endRun: endId});
+ }).then((content) => {
+ return PrivilegedAPI.sendRequest('create-analysis-task', {name: 'hi', startRun: startId, endRun: endId}).then(() => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'DuplicateAnalysisTask');
+ });
+ }).then(() => {
+ return db.selectAll('analysis_tasks');
+ }).then((tasks) => {
+ assert.equal(tasks.length, 1);
+ });
+ });
+
+ it('should create an analysis task with analysis strategies when they are specified', () => {
+ const db = TestServer.database();
+ return addBuilderForReport(reportWithRevision[0]).then(() => {
+ return TestServer.remoteAPI().postJSON('/api/report/', reportWithRevision);
+ }).then(() => {
+ return TestServer.remoteAPI().postJSON('/api/report/', anotherReportWithRevision);
+ }).then(() => {
+ return Manifest.fetch();
+ }).then(() => {
+ const test1 = Test.findByPath(['Suite', 'test1']);
+ const platform = Platform.findByName('some platform');
+ return db.selectFirstRow('test_configurations', {metric: test1.metrics()[0].id(), platform: platform.id()});
+ }).then((configRow) => {
+ return db.selectRows('test_runs', {config: configRow['id']});
+ }).then((testRuns) => {
+ assert.equal(testRuns.length, 2);
+ return PrivilegedAPI.sendRequest('create-analysis-task', {
+ name: 'hi',
+ startRun: testRuns[0]['id'],
+ endRun: testRuns[1]['id'],
+ segmentationStrategy: "time series segmentation",
+ testRangeStrategy: "student's t-test"});
+ }).then(() => {
+ return Promise.all([db.selectFirstRow('analysis_tasks'), db.selectAll('analysis_strategies')]);
+ }).then((results) => {
+ const [taskRow, strategies] = results;
+ assert(taskRow['segmentation']);
+ assert(taskRow['test_range']);
+
+ const strategyIdMap = {};
+ for (let strategy of strategies)
+ strategyIdMap[strategy['id']] = strategy;
+
+ assert.equal(strategyIdMap[taskRow['segmentation']]['name'], "time series segmentation");
+ assert.equal(strategyIdMap[taskRow['test_range']]['name'], "student's t-test");
+ });
+ });
+
+});
</ins></span></pre></div>
<a id="trunkWebsitesperfwebkitorgservertestsprivilegedapicreatetestgrouptestsjs"></a>
<div class="addfile"><h4>Added: trunk/Websites/perf.webkit.org/server-tests/privileged-api-create-test-group-tests.js (0 => 213998)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/server-tests/privileged-api-create-test-group-tests.js (rev 0)
+++ trunk/Websites/perf.webkit.org/server-tests/privileged-api-create-test-group-tests.js 2017-03-15 19:15:02 UTC (rev 213998)
</span><span class="lines">@@ -0,0 +1,291 @@
</span><ins>+'use strict';
+
+let assert = require('assert');
+
+let MockData = require('./resources/mock-data.js');
+let TestServer = require('./resources/test-server.js');
+const addSlaveForReport = require('./resources/common-operations.js').addSlaveForReport;
+const prepareServerTest = require('./resources/common-operations.js').prepareServerTest;
+
+function createAnalysisTask(name)
+{
+ const reportWithRevision = [{
+ "buildNumber": "124",
+ "buildTime": "2015-10-27T15:34:51",
+ "revisions": {
+ "WebKit": {
+ "revision": "191622",
+ "timestamp": '2015-10-27T11:36:56.878473Z',
+ },
+ "macOS": {
+ "revision": "15A284",
+ }
+ },
+ "builderName": "someBuilder",
+ "slaveName": "someSlave",
+ "slavePassword": "somePassword",
+ "platform": "some platform",
+ "tests": {
+ "some test": {
+ "metrics": {
+ "Time": ["Arithmetic"],
+ },
+ "tests": {
+ "test1": {
+ "metrics": {"Time": { "current": [11] }},
+ }
+ }
+ },
+ }}];
+
+ const anotherReportWithRevision = [{
+ "buildNumber": "125",
+ "buildTime": "2015-10-27T17:27:41",
+ "revisions": {
+ "WebKit": {
+ "revision": "191623",
+ "timestamp": '2015-10-27T16:38:10.768995Z',
+ },
+ "macOS": {
+ "revision": "15A284",
+ }
+ },
+ "builderName": "someBuilder",
+ "slaveName": "someSlave",
+ "slavePassword": "somePassword",
+ "platform": "some platform",
+ "tests": {
+ "some test": {
+ "metrics": {
+ "Time": ["Arithmetic"],
+ },
+ "tests": {
+ "test1": {
+ "metrics": {"Time": { "current": [12] }},
+ }
+ }
+ },
+ }}];
+
+ const db = TestServer.database();
+ const remote = TestServer.remoteAPI();
+ return addSlaveForReport(reportWithRevision[0]).then(() => {
+ return remote.postJSON('/api/report/', reportWithRevision);
+ }).then(() => {
+ return remote.postJSON('/api/report/', anotherReportWithRevision);
+ }).then((result) => {
+ return Manifest.fetch();
+ }).then(() => {
+ const test = Test.findByPath(['some test', 'test1']);
+ const platform = Platform.findByName('some platform');
+ return db.selectFirstRow('test_configurations', {metric: test.metrics()[0].id(), platform: platform.id()});
+ }).then((configRow) => {
+ return db.selectRows('test_runs', {config: configRow['id']});
+ }).then((testRuns) => {
+ assert.equal(testRuns.length, 2);
+ return PrivilegedAPI.sendRequest('create-analysis-task', {
+ name: name,
+ startRun: testRuns[0]['id'],
+ endRun: testRuns[1]['id'],
+ });
+ }).then((content) => content['taskId']);
+}
+
+function addTriggerableAndCreateTask(name)
+{
+ const report = {
+ 'slaveName': 'anotherSlave',
+ 'slavePassword': 'anotherPassword',
+ 'triggerable': 'build-webkit',
+ 'configurations': [
+ {test: MockData.someTestId(), platform: MockData.somePlatformId()}
+ ],
+ };
+ return MockData.addMockData(TestServer.database()).then(() => {
+ return addSlaveForReport(report);
+ }).then(() => {
+ return TestServer.remoteAPI().postJSON('/api/update-triggerable/', report);
+ }).then(() => {
+ return createAnalysisTask(name);
+ });
+}
+
+describe('/privileged-api/create-test-group', function () {
+ prepareServerTest(this);
+
+ it('should return "InvalidName" on an empty request', () => {
+ return PrivilegedAPI.sendRequest('create-test-group', {}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidName');
+ });
+ });
+
+ it('should return "InvalidTask" when task is not specified', () => {
+ return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', commitSets: [[1]]}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidTask');
+ });
+ });
+
+ it('should return "InvalidTask" when task is not a valid integer', () => {
+ return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: 'foo', commitSets: [[1]]}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidTask');
+ });
+ });
+
+ it('should return "InvalidCommitSets" when commit sets are not specified', () => {
+ return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: 1, repetitionCount: 1}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidCommitSets');
+ });
+ });
+
+ it('should return "InvalidCommitSets" when commit sets is empty', () => {
+ return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: 1, repetitionCount: 1, commitSets: {}}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidCommitSets');
+ });
+ });
+
+ it('should return "InvalidTask" when there is no matching task', () => {
+ return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: 1, repetitionCount: 1, commitSets: {'WebKit': []}}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidTask');
+ });
+ });
+
+ it('should return "InvalidRepetitionCount" when repetitionCount is not a valid integer', () => {
+ return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: 1, repetitionCount: 'foo', commitSets: {'WebKit': []}}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidRepetitionCount');
+ });
+ });
+
+ it('should return "InvalidRepetitionCount" when repetitionCount is a negative integer', () => {
+ return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: 1, repetitionCount: -5, commitSets: {'WebKit': []}}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidRepetitionCount');
+ });
+ });
+
+ it('should return "InvalidTask" when there is no matching task', () => {
+ return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: 1, commitSets: {'WebKit': []}}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidTask');
+ });
+ });
+
+ it('should return "TriggerableNotFoundForTask" when there is no matching triggerable', () => {
+ return createAnalysisTask('some task').then((taskId) => {
+ return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, commitSets: {'WebKit': []}}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'TriggerableNotFoundForTask');
+ });
+ });
+ });
+
+ it('should return "InvalidCommitSets" when each repository specifies zero revisions', () => {
+ return addTriggerableAndCreateTask('some task').then((taskId) => {
+ return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, commitSets: {'WebKit': []}}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidCommitSets');
+ });
+ });
+ });
+
+ it('should return "RepositoryNotFound" when commit sets contains an invalid repository', () => {
+ return addTriggerableAndCreateTask('some task').then((taskId) => {
+ return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, commitSets: {'Foo': []}}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'RepositoryNotFound');
+ });
+ });
+ });
+
+ it('should return "RevisionNotFound" when commit sets contains an invalid revision', () => {
+ return addTriggerableAndCreateTask('some task').then((taskId) => {
+ return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, commitSets: {'WebKit': ['1']}}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'RevisionNotFound');
+ });
+ });
+ });
+
+ it('should return "InvalidCommitSets" when commit sets contains an inconsistent number of revisions', () => {
+ return addTriggerableAndCreateTask('some task').then((taskId) => {
+ return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, commitSets: {'WebKit': ['191622', '191623'], 'macOS': ['15A284']}}).then((content) => {
+ assert(false, 'should never be reached');
+ }, (response) => {
+ assert.equal(response['status'], 'InvalidCommitSets');
+ });
+ });
+ });
+
+ it('should create a test group with the repetition count of one when repetitionCount is omitted', () => {
+ return addTriggerableAndCreateTask('some task').then((taskId) => {
+ let insertedGroupId;
+ return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, commitSets: {'WebKit': ['191622', '191623']}}).then((content) => {
+ insertedGroupId = content['testGroupId'];
+ return TestGroup.fetchByTask(taskId);
+ }).then((testGroups) => {
+ assert.equal(testGroups.length, 1);
+ const group = testGroups[0];
+ assert.equal(group.id(), insertedGroupId);
+ assert.equal(group.repetitionCount(), 1);
+ const requests = group.buildRequests();
+ assert.equal(requests.length, 2);
+ const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
+ assert.deepEqual(requests[0].commitSet().repositories(), [webkit]);
+ assert.deepEqual(requests[1].commitSet().repositories(), [webkit]);
+ assert.equal(requests[0].commitSet().revisionForRepository(webkit), '191622');
+ assert.equal(requests[1].commitSet().revisionForRepository(webkit), '191623');
+ });
+ });
+ });
+
+ it('should create a test group with the repetition count of two with two repositories', () => {
+ return addTriggerableAndCreateTask('some task').then((taskId) => {
+ let insertedGroupId;
+ return PrivilegedAPI.sendRequest('create-test-group', {name: 'test', task: taskId, repetitionCount: 2,
+ commitSets: {'WebKit': ['191622', '191623'], 'macOS': ['15A284', '15A284']}}).then((content) => {
+ insertedGroupId = content['testGroupId'];
+ return TestGroup.fetchByTask(taskId);
+ }).then((testGroups) => {
+ assert.equal(testGroups.length, 1);
+ const group = testGroups[0];
+ assert.equal(group.id(), insertedGroupId);
+ assert.equal(group.repetitionCount(), 2);
+ const requests = group.buildRequests();
+ assert.equal(requests.length, 4);
+ const webkit = Repository.all().filter((repository) => repository.name() == 'WebKit')[0];
+ const macos = Repository.all().filter((repository) => repository.name() == 'macOS')[0];
+ const set1 = requests[0].commitSet();
+ const set2 = requests[1].commitSet();
+ assert.equal(requests[2].commitSet(), set1);
+ assert.equal(requests[3].commitSet(), set2);
+ assert.deepEqual(Repository.sortByNamePreferringOnesWithURL(set1.repositories()), [webkit, macos]);
+ assert.deepEqual(Repository.sortByNamePreferringOnesWithURL(set2.repositories()), [webkit, macos]);
+ assert.equal(set1.revisionForRepository(webkit), '191622');
+ assert.equal(set1.revisionForRepository(macos), '15A284');
+ assert.equal(set2.revisionForRepository(webkit), '191623');
+ assert.equal(set2.revisionForRepository(macos), '15A284');
+ assert.equal(set1.commitForRepository(macos), set2.commitForRepository(macos));
+ });
+ });
+ });
+
+});
</ins></span></pre></div>
<a id="trunkWebsitesperfwebkitorgservertestsresourcescommonoperationsjs"></a>
<div class="modfile"><h4>Modified: trunk/Websites/perf.webkit.org/server-tests/resources/common-operations.js (213997 => 213998)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/server-tests/resources/common-operations.js 2017-03-15 19:07:55 UTC (rev 213997)
+++ trunk/Websites/perf.webkit.org/server-tests/resources/common-operations.js 2017-03-15 19:15:02 UTC (rev 213998)
</span><span class="lines">@@ -19,7 +19,7 @@
</span><span class="cx">
</span><span class="cx"> function prepareServerTest(test)
</span><span class="cx"> {
</span><del>- test.timeout(1000);
</del><ins>+ test.timeout(5000);
</ins><span class="cx"> TestServer.inject();
</span><span class="cx">
</span><span class="cx"> beforeEach(function () {
</span></span></pre></div>
<a id="trunkWebsitesperfwebkitorgservertestsresourcesmockdatajs"></a>
<div class="modfile"><h4>Modified: trunk/Websites/perf.webkit.org/server-tests/resources/mock-data.js (213997 => 213998)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/server-tests/resources/mock-data.js 2017-03-15 19:07:55 UTC (rev 213997)
+++ trunk/Websites/perf.webkit.org/server-tests/resources/mock-data.js 2017-03-15 19:15:02 UTC (rev 213998)
</span><span class="lines">@@ -25,7 +25,7 @@
</span><span class="cx"> statusList = ['pending', 'pending', 'pending', 'pending'];
</span><span class="cx"> return Promise.all([
</span><span class="cx"> db.insert('build_triggerables', {id: 1, name: 'build-webkit'}),
</span><del>- db.insert('build_slaves', {id: 2, name: 'sync-slave', password_hash: crypto.createHash('sha256').update('password').digest('hex')}),
</del><ins>+ db.insert('build_slaves', {id: 20, name: 'sync-slave', password_hash: crypto.createHash('sha256').update('password').digest('hex')}),
</ins><span class="cx"> db.insert('repositories', {id: 9, name: 'OS X'}),
</span><span class="cx"> db.insert('repositories', {id: 11, name: 'WebKit'}),
</span><span class="cx"> db.insert('commits', {id: 87832, repository: 9, revision: '10.11 15A284'}),
</span></span></pre>
</div>
</div>
</body>
</html>