<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[177530] trunk/Websites/perf.webkit.org</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/177530">177530</a></dd>
<dt>Author</dt> <dd>rniwa@webkit.org</dd>
<dt>Date</dt> <dd>2014-12-18 14:53:52 -0800 (Thu, 18 Dec 2014)</dd>
</dl>
<h3>Log Message</h3>
<pre>New perf dashboard should not duplicate author information in each commit
https://bugs.webkit.org/show_bug.cgi?id=139756
Reviewed by Darin Adler.
Instead of each commit having author name and email, make it reference a newly added committers table.
Also replace "email" by "account" since some repositories don't use emails as account names.
This improves the keyword search performance in commits.php since LIKE now runs on committers table,
which only contains as many rows as there are accounts in each repository, instead of commits table
which contains every commit ever happened in each repository.
To migrate an existing database into match the new schema, run:
BEGIN;
INSERT INTO committers (committer_repository, committer_name, committer_email)
(SELECT DISTINCT commit_repository, commit_author_name, commit_author_email
FROM commits WHERE commit_author_email IS NOT NULL);
ALTER TABLE commits ADD COLUMN commit_committer integer REFERENCES committers ON DELETE CASCADE;
UPDATE commits SET commit_committer = committer_id FROM committers
WHERE commit_repository = committer_repository AND commit_author_email = committer_email;
ALTER TABLE commits DROP COLUMN commit_author_name CASCADE;
ALTER TABLE commits DROP COLUMN commit_author_email CASCADE;
COMMIT;
* init-database.sql: Added committers table, and replaced author columns in commits table by a foreign
reference to committers. Also added the missing drop table statements.
* public/api/commits.php:
(main): Fetch the corresponding committers row for a single commit. Also wrap a single commit by
an array here instead of doing it in format_commit.
(fetch_commits_between): Updated queries to JOIN commits with committers.
(format_commit): Takes both commit and committers rows. Also don't wrap the result in an array as that
is now done in main.
* public/api/report-commits.php:
(main): Store the reported committer information or update the existing entry if there is one.
* tests/admin-regenerate-manifest.js: Fixed tests.
* tests/api-report-commits.js: Ditto. Also added a test for updating an existing committer entry.
* tools/pull-svn.py: Renamed email to account.
(main):
(fetch_commit_and_resolve_author):
(fetch_commit):
(resolve_author_name_from_account):
(resolve_author_name_from_email): Deleted.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkWebsitesperfwebkitorgChangeLog">trunk/Websites/perf.webkit.org/ChangeLog</a></li>
<li><a href="#trunkWebsitesperfwebkitorginitdatabasesql">trunk/Websites/perf.webkit.org/init-database.sql</a></li>
<li><a href="#trunkWebsitesperfwebkitorgpublicapicommitsphp">trunk/Websites/perf.webkit.org/public/api/commits.php</a></li>
<li><a href="#trunkWebsitesperfwebkitorgpublicapireportcommitsphp">trunk/Websites/perf.webkit.org/public/api/report-commits.php</a></li>
<li><a href="#trunkWebsitesperfwebkitorgtestsadminregeneratemanifestjs">trunk/Websites/perf.webkit.org/tests/admin-regenerate-manifest.js</a></li>
<li><a href="#trunkWebsitesperfwebkitorgtestsapireportcommitsjs">trunk/Websites/perf.webkit.org/tests/api-report-commits.js</a></li>
<li><a href="#trunkWebsitesperfwebkitorgtoolspullsvnpy">trunk/Websites/perf.webkit.org/tools/pull-svn.py</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkWebsitesperfwebkitorgChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Websites/perf.webkit.org/ChangeLog (177529 => 177530)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/ChangeLog 2014-12-18 22:51:09 UTC (rev 177529)
+++ trunk/Websites/perf.webkit.org/ChangeLog 2014-12-18 22:53:52 UTC (rev 177530)
</span><span class="lines">@@ -1,3 +1,59 @@
</span><ins>+2014-12-18 Ryosuke Niwa <rniwa@webkit.org>
+
+ New perf dashboard should not duplicate author information in each commit
+ https://bugs.webkit.org/show_bug.cgi?id=139756
+
+ Reviewed by Darin Adler.
+
+ Instead of each commit having author name and email, make it reference a newly added committers table.
+ Also replace "email" by "account" since some repositories don't use emails as account names.
+
+ This improves the keyword search performance in commits.php since LIKE now runs on committers table,
+ which only contains as many rows as there are accounts in each repository, instead of commits table
+ which contains every commit ever happened in each repository.
+
+ To migrate an existing database into match the new schema, run:
+
+ BEGIN;
+
+ INSERT INTO committers (committer_repository, committer_name, committer_email)
+ (SELECT DISTINCT commit_repository, commit_author_name, commit_author_email
+ FROM commits WHERE commit_author_email IS NOT NULL);
+
+ ALTER TABLE commits ADD COLUMN commit_committer integer REFERENCES committers ON DELETE CASCADE;
+
+ UPDATE commits SET commit_committer = committer_id FROM committers
+ WHERE commit_repository = committer_repository AND commit_author_email = committer_email;
+
+ ALTER TABLE commits DROP COLUMN commit_author_name CASCADE;
+ ALTER TABLE commits DROP COLUMN commit_author_email CASCADE;
+
+ COMMIT;
+
+ * init-database.sql: Added committers table, and replaced author columns in commits table by a foreign
+ reference to committers. Also added the missing drop table statements.
+
+ * public/api/commits.php:
+ (main): Fetch the corresponding committers row for a single commit. Also wrap a single commit by
+ an array here instead of doing it in format_commit.
+ (fetch_commits_between): Updated queries to JOIN commits with committers.
+ (format_commit): Takes both commit and committers rows. Also don't wrap the result in an array as that
+ is now done in main.
+
+ * public/api/report-commits.php:
+ (main): Store the reported committer information or update the existing entry if there is one.
+
+ * tests/admin-regenerate-manifest.js: Fixed tests.
+
+ * tests/api-report-commits.js: Ditto. Also added a test for updating an existing committer entry.
+
+ * tools/pull-svn.py: Renamed email to account.
+ (main):
+ (fetch_commit_and_resolve_author):
+ (fetch_commit):
+ (resolve_author_name_from_account):
+ (resolve_author_name_from_email): Deleted.
+
</ins><span class="cx"> 2014-12-17 Ryosuke Niwa <rniwa@webkit.org>
</span><span class="cx">
</span><span class="cx"> Unreviewed build fix.
</span></span></pre></div>
<a id="trunkWebsitesperfwebkitorginitdatabasesql"></a>
<div class="modfile"><h4>Modified: trunk/Websites/perf.webkit.org/init-database.sql (177529 => 177530)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/init-database.sql 2014-12-18 22:51:09 UTC (rev 177529)
+++ trunk/Websites/perf.webkit.org/init-database.sql 2014-12-18 22:53:52 UTC (rev 177530)
</span><span class="lines">@@ -4,6 +4,7 @@
</span><span class="cx"> DROP TYPE test_configuration_type CASCADE;
</span><span class="cx"> DROP TABLE aggregators CASCADE;
</span><span class="cx"> DROP TABLE builds CASCADE;
</span><ins>+DROP TABLE committers CASCADE;
</ins><span class="cx"> DROP TABLE commits CASCADE;
</span><span class="cx"> DROP TABLE build_commits CASCADE;
</span><span class="cx"> DROP TABLE builders CASCADE;
</span><span class="lines">@@ -14,7 +15,13 @@
</span><span class="cx"> DROP TABLE reports CASCADE;
</span><span class="cx"> DROP TABLE tracker_repositories CASCADE;
</span><span class="cx"> DROP TABLE bug_trackers CASCADE;
</span><ins>+DROP TABLE analysis_tasks CASCADE;
+DROP TABLE bugs CASCADE;
+DROP TABLE analysis_test_groups CASCADE;
+DROP TABLE root_sets CASCADE;
+DROP TABLE build_requests CASCADE;
</ins><span class="cx">
</span><ins>+
</ins><span class="cx"> CREATE TABLE platforms (
</span><span class="cx"> platform_id serial PRIMARY KEY,
</span><span class="cx"> platform_name varchar(64) NOT NULL,
</span><span class="lines">@@ -51,20 +58,26 @@
</span><span class="cx"> CONSTRAINT builder_build_time_tuple_must_be_unique UNIQUE(build_builder, build_number, build_time));
</span><span class="cx"> CREATE INDEX build_builder_index ON builds(build_builder);
</span><span class="cx">
</span><ins>+CREATE TABLE committers (
+ committer_id serial PRIMARY KEY,
+ committer_repository integer NOT NULL REFERENCES repositories ON DELETE CASCADE,
+ committer_account varchar(320) NOT NULL,
+ committer_name varchar(128),
+ CONSTRAINT committer_in_repository_must_be_unique UNIQUE(committer_repository, committer_account));
+CREATE INDEX committer_account_index ON committers(committer_account);
+CREATE INDEX committer_name_index ON committers(committer_name);
+
</ins><span class="cx"> CREATE TABLE commits (
</span><span class="cx"> commit_id serial PRIMARY KEY,
</span><span class="cx"> commit_repository integer NOT NULL REFERENCES repositories ON DELETE CASCADE,
</span><span class="cx"> commit_revision varchar(64) NOT NULL,
</span><span class="cx"> commit_parent integer REFERENCES commits ON DELETE CASCADE,
</span><span class="cx"> commit_time timestamp,
</span><del>- commit_author_name varchar(128),
- commit_author_email varchar(320),
</del><ins>+ commit_committer integer REFERENCES committers ON DELETE CASCADE,
</ins><span class="cx"> commit_message text,
</span><span class="cx"> commit_reported boolean NOT NULL DEFAULT FALSE,
</span><span class="cx"> CONSTRAINT commit_in_repository_must_be_unique UNIQUE(commit_repository, commit_revision));
</span><span class="cx"> CREATE INDEX commit_time_index ON commits(commit_time);
</span><del>-CREATE INDEX commit_author_name_index ON commits(commit_author_name);
-CREATE INDEX commit_author_email_index ON commits(commit_author_email);
</del><span class="cx">
</span><span class="cx"> CREATE TABLE build_commits (
</span><span class="cx"> commit_build integer NOT NULL REFERENCES builds ON DELETE CASCADE,
</span></span></pre></div>
<a id="trunkWebsitesperfwebkitorgpublicapicommitsphp"></a>
<div class="modfile"><h4>Modified: trunk/Websites/perf.webkit.org/public/api/commits.php (177529 => 177530)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/public/api/commits.php 2014-12-18 22:51:09 UTC (rev 177529)
+++ trunk/Websites/perf.webkit.org/public/api/commits.php 2014-12-18 22:53:52 UTC (rev 177530)
</span><span class="lines">@@ -40,7 +40,12 @@
</span><span class="cx"> $commits = fetch_commits_between($db, $repository_id, $matches[1], $matches[2]);
</span><span class="cx"> }
</span><span class="cx">
</span><del>- exit_with_success(array('commits' => $single_commit ? format_commit($single_commit) : $commits));
</del><ins>+ if ($single_commit) {
+ $committer = $db->select_first_row('committers', 'committer', array('id' => $single_commit['commit_committer']));
+ exit_with_success(array('commits' => array(format_commit($single_commit, $committer))));
+ }
+
+ exit_with_success(array('commits' => $commits));
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> function commit_from_revision($db, $repository_id, $revision) {
</span><span class="lines">@@ -59,10 +64,11 @@
</span><span class="cx"> commit_revision as "revision",
</span><span class="cx"> commit_parent as "parent",
</span><span class="cx"> commit_time as "time",
</span><del>- commit_author_name as "authorName",
- commit_author_email as "authorEmail",
</del><ins>+ committer_name as "authorName",
+ committer_account as "authorEmail",
</ins><span class="cx"> commit_message as "message"
</span><del>- FROM commits WHERE commit_repository = $1 AND commit_reported = true';
</del><ins>+ FROM commits JOIN committers ON commit_committer = committer_id
+ WHERE commit_repository = $1 AND commit_reported = true';
</ins><span class="cx"> $values = array($repository_id);
</span><span class="cx">
</span><span class="cx"> if ($first && $second) {
</span><span class="lines">@@ -78,11 +84,12 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> if ($keyword) {
</span><del>- array_push($values, '%' . str_replace(array('\\', '_', '@'), array('\\\\', '\\_', '\\%'), $keyword) . '%');
- $index = '$' . count($values);
- $statements .= " AND (commit_author_name LIKE $index OR commit_author_email LIKE $index";
</del><ins>+ array_push($values, '%' . str_replace(array('\\', '_', '%'), array('\\\\', '\\_', '\\%'), $keyword) . '%');
+ $keyword_index = '$' . count($values);
</ins><span class="cx"> array_push($values, ltrim($keyword, 'r'));
</span><del>- $statements .= ' OR commit_revision = $' . count($values) . ')';
</del><ins>+ $revision_index = '$' . count($values);
+ $statements .= "
+ AND ((committer_name LIKE $keyword_index OR committer_account LIKE $keyword_index) OR commit_revision = $revision_index)";
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> $commits = $db->query_and_fetch_all($statements . ' ORDER BY commit_time', $values);
</span><span class="lines">@@ -91,16 +98,16 @@
</span><span class="cx"> return $commits;
</span><span class="cx"> }
</span><span class="cx">
</span><del>-function format_commit($commit_row) {
- return array(array(
</del><ins>+function format_commit($commit_row, $committer_row) {
+ return array(
</ins><span class="cx"> 'id' => $commit_row['commit_id'],
</span><span class="cx"> 'revision' => $commit_row['commit_revision'],
</span><span class="cx"> 'parent' => $commit_row['commit_parent'],
</span><span class="cx"> 'time' => $commit_row['commit_time'],
</span><del>- 'authorName' => $commit_row['commit_author_name'],
- 'authorEmail' => $commit_row['commit_author_email'],
</del><ins>+ 'authorName' => $committer_row ? $committer_row['committer_name'] : null,
+ 'authorEmail' => $committer_row ? $committer_row['committer_account'] : null,
</ins><span class="cx"> 'message' => $commit_row['commit_message']
</span><del>- ));
</del><ins>+ );
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> main(array_key_exists('PATH_INFO', $_SERVER) ? explode('/', trim($_SERVER['PATH_INFO'], '/')) : array());
</span></span></pre></div>
<a id="trunkWebsitesperfwebkitorgpublicapireportcommitsphp"></a>
<div class="modfile"><h4>Modified: trunk/Websites/perf.webkit.org/public/api/report-commits.php (177529 => 177530)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/public/api/report-commits.php 2014-12-18 22:51:09 UTC (rev 177529)
+++ trunk/Websites/perf.webkit.org/public/api/report-commits.php 2014-12-18 22:53:52 UTC (rev 177530)
</span><span class="lines">@@ -34,6 +34,18 @@
</span><span class="cx"> exit_with_error('FailedToInsertRepository', array('commit' => $commit_info));
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+ $account = array_get($commit_info['author'], 'account');
+ $committer_query = array('repository' => $repository_id, 'account' => $account);
+ $committer_data = $committer_query;
+ $name = array_get($commit_info['author'], 'name');
+ if ($name)
+ $committer_data['name'] = $name;
+ $committer_id = $db->update_or_insert_row('committers', 'committer', $committer_query, $committer_data);
+ if (!$committer_id) {
+ $db->rollback_transaction();
+ exit_with_error('FailedToInsertCommitter', array('committer' => $committer_data));
+ }
+
</ins><span class="cx"> $parent_revision = array_get($commit_info, 'parent');
</span><span class="cx"> $parent_id = NULL;
</span><span class="cx"> if ($parent_revision) {
</span><span class="lines">@@ -50,8 +62,7 @@
</span><span class="cx"> 'revision' => $commit_info['revision'],
</span><span class="cx"> 'parent' => $parent_id,
</span><span class="cx"> 'time' => $commit_info['time'],
</span><del>- 'author_name' => array_get($commit_info['author'], 'name'),
- 'author_email' => array_get($commit_info['author'], 'email'),
</del><ins>+ 'committer' => $committer_id,
</ins><span class="cx"> 'message' => $commit_info['message'],
</span><span class="cx"> 'reported' => true,
</span><span class="cx"> );
</span></span></pre></div>
<a id="trunkWebsitesperfwebkitorgtestsadminregeneratemanifestjs"></a>
<div class="modfile"><h4>Modified: trunk/Websites/perf.webkit.org/tests/admin-regenerate-manifest.js (177529 => 177530)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/tests/admin-regenerate-manifest.js 2014-12-18 22:51:09 UTC (rev 177529)
+++ trunk/Websites/perf.webkit.org/tests/admin-regenerate-manifest.js 2014-12-18 22:53:52 UTC (rev 177530)
</span><span class="lines">@@ -6,6 +6,7 @@
</span><span class="cx"> httpGet('/data/manifest', function (response) {
</span><span class="cx"> assert.equal(response.statusCode, 200);
</span><span class="cx"> var manifest = JSON.parse(response.responseText);
</span><ins>+ delete manifest.defaultDashboard;
</ins><span class="cx"> assert.deepEqual(manifest, {
</span><span class="cx"> all: [],
</span><span class="cx"> bugTrackers: [],
</span><span class="lines">@@ -27,7 +28,8 @@
</span><span class="cx"> httpGet('/data/manifest', function (response) {
</span><span class="cx"> assert.equal(response.statusCode, 200);
</span><span class="cx"> var manifest = JSON.parse(response.responseText);
</span><del>- assert.deepEqual(manifest['bugTrackers'], { 'Bugzilla': { newBugUrl: 'bugs.webkit.org', repositories: null } });
</del><ins>+ assert.deepEqual(manifest['bugTrackers'],
+ {1: {name: 'Bugzilla', bugUrl: null, newBugUrl: 'bugs.webkit.org', repositories: null}});
</ins><span class="cx"> notifyDone();
</span><span class="cx"> });
</span><span class="cx"> });
</span><span class="lines">@@ -48,8 +50,8 @@
</span><span class="cx"> 'WebKit': { url: 'trac.webkit.org', blameUrl: null, hasReportedCommits: false },
</span><span class="cx"> 'Chromium': { url: null, blameUrl: 'SomeBlameURL', hasReportedCommits: false }
</span><span class="cx"> });
</span><del>- assert.deepEqual(manifest['bugTrackers']['Bugzilla'], { newBugUrl: null, repositories: ['WebKit'] });
- assert.deepEqual(manifest['bugTrackers']['Issue Tracker'], { newBugUrl: null, repositories: ['WebKit', 'Chromium'] });
</del><ins>+ assert.deepEqual(manifest['bugTrackers'][3], {name: 'Bugzilla', bugUrl: null, newBugUrl: null, repositories: ['WebKit']});
+ assert.deepEqual(manifest['bugTrackers'][4], {name: 'Issue Tracker', bugUrl: null, newBugUrl: null, repositories: ['WebKit', 'Chromium']});
</ins><span class="cx"> notifyDone();
</span><span class="cx"> });
</span><span class="cx"> });
</span></span></pre></div>
<a id="trunkWebsitesperfwebkitorgtestsapireportcommitsjs"></a>
<div class="modfile"><h4>Modified: trunk/Websites/perf.webkit.org/tests/api-report-commits.js (177529 => 177530)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/tests/api-report-commits.js 2014-12-18 22:51:09 UTC (rev 177529)
+++ trunk/Websites/perf.webkit.org/tests/api-report-commits.js 2014-12-18 22:53:52 UTC (rev 177530)
</span><span class="lines">@@ -11,7 +11,7 @@
</span><span class="cx"> "repository": "WebKit",
</span><span class="cx"> "revision": "141977",
</span><span class="cx"> "time": "2013-02-06T08:55:20.9Z",
</span><del>- "author": {"name": "Commit Queue", "email": "commit-queue@webkit.org"},
</del><ins>+ "author": {"name": "Commit Queue", "account": "commit-queue@webkit.org"},
</ins><span class="cx"> "message": "some message",
</span><span class="cx"> }
</span><span class="cx"> ],
</span><span class="lines">@@ -24,7 +24,7 @@
</span><span class="cx"> "repository": "WebKit",
</span><span class="cx"> "revision": "_141977",
</span><span class="cx"> "time": "2013-02-06T08:55:20.9Z",
</span><del>- "author": {"name": "Commit Queue", "email": "commit-queue@webkit.org"},
</del><ins>+ "author": {"name": "Commit Queue", "account": "commit-queue@webkit.org"},
</ins><span class="cx"> "message": "some message",
</span><span class="cx"> }
</span><span class="cx"> ],
</span><span class="lines">@@ -37,7 +37,7 @@
</span><span class="cx"> "repository": "WebKit",
</span><span class="cx"> "revision": "141977",
</span><span class="cx"> "time": "2013-02-06T08:55:20.9Z",
</span><del>- "author": {"name": "Commit Queue", "email": "commit-queue@webkit.org"},
</del><ins>+ "author": {"name": "Commit Queue", "account": "commit-queue@webkit.org"},
</ins><span class="cx"> "message": "some message",
</span><span class="cx"> },
</span><span class="cx"> {
</span><span class="lines">@@ -45,7 +45,7 @@
</span><span class="cx"> "parent": "141977",
</span><span class="cx"> "revision": "141978",
</span><span class="cx"> "time": "2013-02-06T09:54:56.0Z",
</span><del>- "author": {"name": "Mikhail Pozdnyakov", "email": "mikhail.pozdnyakov@intel.com"},
</del><ins>+ "author": {"name": "Mikhail Pozdnyakov", "account": "mikhail.pozdnyakov@intel.com"},
</ins><span class="cx"> "message": "another message",
</span><span class="cx"> }
</span><span class="cx"> ]
</span><span class="lines">@@ -105,13 +105,13 @@
</span><span class="cx"> postJSON('/api/report-commits/', subversionCommit, function (response) {
</span><span class="cx"> assert.equal(response.statusCode, 200);
</span><span class="cx"> assert.equal(JSON.parse(response.responseText)['status'], 'OK');
</span><del>- queryAndFetchAll('SELECT * FROM commits', [], function (rows) {
</del><ins>+ queryAndFetchAll('SELECT * FROM commits JOIN committers ON commit_committer = committer_id', [], function (rows) {
</ins><span class="cx"> assert.equal(rows.length, 1);
</span><span class="cx"> var reportedData = subversionCommit.commits[0];
</span><span class="cx"> assert.equal(rows[0]['commit_revision'], reportedData['revision']);
</span><span class="cx"> assert.equal(rows[0]['commit_time'].toString(), new Date('2013-02-06 08:55:20.9').toString());
</span><del>- assert.equal(rows[0]['commit_author_name'], reportedData['author']['name']);
- assert.equal(rows[0]['commit_author_email'], reportedData['author']['email']);
</del><ins>+ assert.equal(rows[0]['committer_name'], reportedData['author']['name']);
+ assert.equal(rows[0]['committer_account'], reportedData['author']['account']);
</ins><span class="cx"> assert.equal(rows[0]['commit_message'], reportedData['message']);
</span><span class="cx"> notifyDone();
</span><span class="cx"> });
</span><span class="lines">@@ -138,19 +138,19 @@
</span><span class="cx"> postJSON('/api/report-commits/', subversionTwoCommits, function (response) {
</span><span class="cx"> assert.equal(response.statusCode, 200);
</span><span class="cx"> assert.equal(JSON.parse(response.responseText)['status'], 'OK');
</span><del>- queryAndFetchAll('SELECT * FROM commits ORDER BY commit_time', [], function (rows) {
</del><ins>+ queryAndFetchAll('SELECT * FROM commits JOIN committers ON commit_committer = committer_id ORDER BY commit_time', [], function (rows) {
</ins><span class="cx"> assert.equal(rows.length, 2);
</span><span class="cx"> var reportedData = subversionTwoCommits.commits[0];
</span><span class="cx"> assert.equal(rows[0]['commit_revision'], reportedData['revision']);
</span><span class="cx"> assert.equal(rows[0]['commit_time'].toString(), new Date('2013-02-06 08:55:20.9').toString());
</span><del>- assert.equal(rows[0]['commit_author_name'], reportedData['author']['name']);
- assert.equal(rows[0]['commit_author_email'], reportedData['author']['email']);
</del><ins>+ assert.equal(rows[0]['committer_name'], reportedData['author']['name']);
+ assert.equal(rows[0]['committer_account'], reportedData['author']['account']);
</ins><span class="cx"> assert.equal(rows[0]['commit_message'], reportedData['message']);
</span><span class="cx"> var reportedData = subversionTwoCommits.commits[1];
</span><span class="cx"> assert.equal(rows[1]['commit_revision'], reportedData['revision']);
</span><span class="cx"> assert.equal(rows[1]['commit_time'].toString(), new Date('2013-02-06 09:54:56.0').toString());
</span><del>- assert.equal(rows[1]['commit_author_name'], reportedData['author']['name']);
- assert.equal(rows[1]['commit_author_email'], reportedData['author']['email']);
</del><ins>+ assert.equal(rows[1]['committer_name'], reportedData['author']['name']);
+ assert.equal(rows[1]['committer_account'], reportedData['author']['account']);
</ins><span class="cx"> assert.equal(rows[1]['commit_message'], reportedData['message']);
</span><span class="cx"> notifyDone();
</span><span class="cx"> });
</span><span class="lines">@@ -165,18 +165,16 @@
</span><span class="cx"> queryAndFetchAll('INSERT INTO commits (commit_repository, commit_revision, commit_time) VALUES ($1, $2, $3) RETURNING *',
</span><span class="cx"> [repositoryId, reportedData['revision'], reportedData['time']], function (existingCommits) {
</span><span class="cx"> var commitId = existingCommits[0]['commit_id'];
</span><del>- assert.equal(existingCommits[0]['commit_author_name'], null);
- assert.equal(existingCommits[0]['commit_author_email'], null);
</del><span class="cx"> assert.equal(existingCommits[0]['commit_message'], null);
</span><span class="cx"> addBuilder(subversionCommit, function () {
</span><span class="cx"> postJSON('/api/report-commits/', subversionCommit, function (response) {
</span><span class="cx"> assert.equal(response.statusCode, 200);
</span><span class="cx"> assert.equal(JSON.parse(response.responseText)['status'], 'OK');
</span><del>- queryAndFetchAll('SELECT * FROM commits', [], function (rows) {
</del><ins>+ queryAndFetchAll('SELECT * FROM commits JOIN committers ON commit_committer = committer_id', [], function (rows) {
</ins><span class="cx"> assert.equal(rows.length, 1);
</span><span class="cx"> var reportedData = subversionCommit.commits[0];
</span><del>- assert.equal(rows[0]['commit_author_name'], reportedData['author']['name']);
- assert.equal(rows[0]['commit_author_email'], reportedData['author']['email']);
</del><ins>+ assert.equal(rows[0]['committer_name'], reportedData['author']['name']);
+ assert.equal(rows[0]['committer_account'], reportedData['author']['account']);
</ins><span class="cx"> assert.equal(rows[0]['commit_message'], reportedData['message']);
</span><span class="cx"> notifyDone();
</span><span class="cx"> });
</span><span class="lines">@@ -199,13 +197,13 @@
</span><span class="cx"> postJSON('/api/report-commits/', subversionCommit, function (response) {
</span><span class="cx"> assert.equal(response.statusCode, 200);
</span><span class="cx"> assert.equal(JSON.parse(response.responseText)['status'], 'OK');
</span><del>- queryAndFetchAll('SELECT * FROM commits ORDER BY commit_time', [], function (rows) {
</del><ins>+ queryAndFetchAll('SELECT * FROM commits LEFT OUTER JOIN committers ON commit_committer = committer_id ORDER BY commit_time', [], function (rows) {
</ins><span class="cx"> assert.equal(rows.length, 2);
</span><del>- assert.equal(rows[0]['commit_author_name'], reportedData['author']['name']);
- assert.equal(rows[0]['commit_author_email'], reportedData['author']['email']);
</del><ins>+ assert.equal(rows[0]['committer_name'], reportedData['author']['name']);
+ assert.equal(rows[0]['committer_account'], reportedData['author']['account']);
</ins><span class="cx"> assert.equal(rows[0]['commit_message'], reportedData['message']);
</span><del>- assert.equal(rows[1]['commit_author_name'], null);
- assert.equal(rows[1]['commit_author_email'], null);
</del><ins>+ assert.equal(rows[1]['committer_name'], null);
+ assert.equal(rows[1]['committer_account'], null);
</ins><span class="cx"> assert.equal(rows[1]['commit_message'], null);
</span><span class="cx"> notifyDone();
</span><span class="cx"> });
</span><span class="lines">@@ -216,4 +214,24 @@
</span><span class="cx"> });
</span><span class="cx"> });
</span><span class="cx">
</span><ins>+ it("should update an existing committer if there is one", function () {
+ queryAndFetchAll('INSERT INTO repositories (repository_id, repository_name) VALUES (1, \'WebKit\')', [], function () {
+ var author = subversionCommit.commits[0]['author'];
+ queryAndFetchAll('INSERT INTO committers (committer_repository, committer_account) VALUES (1, $1)', [author['account']], function () {
+ addBuilder(subversionCommit, function () {
+ postJSON('/api/report-commits/', subversionCommit, function (response) {
+ assert.equal(response.statusCode, 200);
+ assert.equal(JSON.parse(response.responseText)['status'], 'OK');
+ queryAndFetchAll('SELECT * FROM committers', [], function (rows) {
+ assert.equal(rows.length, 1);
+ assert.equal(rows[0]['committer_name'], author['name']);
+ assert.equal(rows[0]['committer_account'], author['account']);
+ notifyDone();
+ });
+ });
+ });
+ });
+ });
+ });
+
</ins><span class="cx"> });
</span></span></pre></div>
<a id="trunkWebsitesperfwebkitorgtoolspullsvnpy"></a>
<div class="modfile"><h4>Modified: trunk/Websites/perf.webkit.org/tools/pull-svn.py (177529 => 177530)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/tools/pull-svn.py 2014-12-18 22:51:09 UTC (rev 177529)
+++ trunk/Websites/perf.webkit.org/tools/pull-svn.py 2014-12-18 22:53:52 UTC (rev 177530)
</span><span class="lines">@@ -12,7 +12,7 @@
</span><span class="cx">
</span><span class="cx"> def main(argv):
</span><span class="cx"> if len(argv) < 7:
</span><del>- sys.exit('Usage: pull-svn <repository-name> <repository-URL> <dashboard-URL> <builder-name> <builder-password> <seconds-to-sleep> [<email-to-name-helper>]')
</del><ins>+ sys.exit('Usage: pull-svn <repository-name> <repository-URL> <dashboard-URL> <builder-name> <builder-password> <seconds-to-sleep> [<account-to-name-helper>]')
</ins><span class="cx">
</span><span class="cx"> repository_name = argv[1]
</span><span class="cx"> repository_url = argv[2]
</span><span class="lines">@@ -20,7 +20,7 @@
</span><span class="cx"> builder_name = argv[4]
</span><span class="cx"> builder_password = argv[5]
</span><span class="cx"> seconds_to_sleep = float(argv[6])
</span><del>- email_to_name_helper = argv[7] if len(argv) > 7 else None
</del><ins>+ account_to_name_helper = argv[7] if len(argv) > 7 else None
</ins><span class="cx">
</span><span class="cx"> print "Submitting revision logs for %s at %s to %s" % (repository_name, repository_url, dashboard_url)
</span><span class="cx">
</span><span class="lines">@@ -30,7 +30,7 @@
</span><span class="cx"> pending_commits_to_send = []
</span><span class="cx">
</span><span class="cx"> while True:
</span><del>- commit = fetch_commit_and_resolve_author(repository_name, repository_url, email_to_name_helper, revision_to_fetch)
</del><ins>+ commit = fetch_commit_and_resolve_author(repository_name, repository_url, account_to_name_helper, revision_to_fetch)
</ins><span class="cx">
</span><span class="cx"> if commit:
</span><span class="cx"> print "Fetched r%d." % revision_to_fetch
</span><span class="lines">@@ -74,7 +74,7 @@
</span><span class="cx"> return int(commits[0]['revision']) if commits else None
</span><span class="cx">
</span><span class="cx">
</span><del>-def fetch_commit_and_resolve_author(repository_name, repository_url, email_to_name_helper, revision_to_fetch):
</del><ins>+def fetch_commit_and_resolve_author(repository_name, repository_url, account_to_name_helper, revision_to_fetch):
</ins><span class="cx"> try:
</span><span class="cx"> commit = fetch_commit(repository_name, repository_url, revision_to_fetch)
</span><span class="cx"> except Exception as error:
</span><span class="lines">@@ -83,13 +83,13 @@
</span><span class="cx"> if not commit:
</span><span class="cx"> return None
</span><span class="cx">
</span><del>- email = commit['author']['email']
</del><ins>+ account = commit['author']['account']
</ins><span class="cx"> try:
</span><del>- name = resolve_author_name_from_email(email_to_name_helper, email) if email_to_name_helper else None
</del><ins>+ name = resolve_author_name_from_account(account_to_name_helper, account) if account_to_name_helper else None
</ins><span class="cx"> if name:
</span><span class="cx"> commit['author']['name'] = name
</span><span class="cx"> except Exception as error:
</span><del>- sys.exit('Failed to resolve the author name from an email %s: %s' % (email, str(error)))
</del><ins>+ sys.exit('Failed to resolve the author name from an account %s: %s' % (account, str(error)))
</ins><span class="cx">
</span><span class="cx"> return commit
</span><span class="cx">
</span><span class="lines">@@ -104,13 +104,13 @@
</span><span class="cx"> raise error
</span><span class="cx"> xml = parseXmlString(output)
</span><span class="cx"> time = textContent(xml.getElementsByTagName("date")[0])
</span><del>- author_email = textContent(xml.getElementsByTagName("author")[0])
</del><ins>+ author_account = textContent(xml.getElementsByTagName("author")[0])
</ins><span class="cx"> message = textContent(xml.getElementsByTagName("msg")[0])
</span><span class="cx"> return {
</span><span class="cx"> 'repository': repository_name,
</span><span class="cx"> 'revision': revision,
</span><span class="cx"> 'time': time,
</span><del>- 'author': {'email': author_email},
</del><ins>+ 'author': {'account': author_account},
</ins><span class="cx"> 'message': message,
</span><span class="cx"> }
</span><span class="cx">
</span><span class="lines">@@ -125,12 +125,12 @@
</span><span class="cx"> return text
</span><span class="cx">
</span><span class="cx">
</span><del>-name_email_compound_regex = re.compile(r'^\s*(?P<name>(\".+\"|[^<]+?))\s*\<(?P<email>.+)\>\s*$')
</del><ins>+name_account_compound_regex = re.compile(r'^\s*(?P<name>(\".+\"|[^<]+?))\s*\<(?P<account>.+)\>\s*$')
</ins><span class="cx">
</span><span class="cx">
</span><del>-def resolve_author_name_from_email(helper, email):
- output = subprocess.check_output(helper + ' ' + email, shell=True)
- match = name_email_compound_regex.match(output)
</del><ins>+def resolve_author_name_from_account(helper, account):
+ output = subprocess.check_output(helper + ' ' + account, shell=True)
+ match = name_account_compound_regex.match(output)
</ins><span class="cx"> if match:
</span><span class="cx"> return match.group('name').strip('"')
</span><span class="cx"> return output.strip()
</span></span></pre>
</div>
</div>
</body>
</html>