<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[212421] trunk/Websites/perf.webkit.org</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/212421">212421</a></dd>
<dt>Author</dt> <dd>rniwa@webkit.org</dd>
<dt>Date</dt> <dd>2017-02-15 21:00:25 -0800 (Wed, 15 Feb 2017)</dd>
</dl>
<h3>Log Message</h3>
<pre>Update ReadMe.md and merge it with Install.md
https://bugs.webkit.org/show_bug.cgi?id=168405
Reviewed by Michael Catanzaro.
Merged Install.md and ReadMe.md into one file.
* Install.md: Removed.
* ReadMe.md: Merged Install.md at the top and updated the rest of the content.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkWebsitesperfwebkitorgChangeLog">trunk/Websites/perf.webkit.org/ChangeLog</a></li>
<li><a href="#trunkWebsitesperfwebkitorgReadMemd">trunk/Websites/perf.webkit.org/ReadMe.md</a></li>
</ul>
<h3>Removed Paths</h3>
<ul>
<li><a href="#trunkWebsitesperfwebkitorgInstallmd">trunk/Websites/perf.webkit.org/Install.md</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkWebsitesperfwebkitorgChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Websites/perf.webkit.org/ChangeLog (212420 => 212421)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/ChangeLog 2017-02-16 04:25:22 UTC (rev 212420)
+++ trunk/Websites/perf.webkit.org/ChangeLog 2017-02-16 05:00:25 UTC (rev 212421)
</span><span class="lines">@@ -1,3 +1,15 @@
</span><ins>+2017-02-15 Ryosuke Niwa <rniwa@webkit.org>
+
+ Update ReadMe.md and merge it with Install.md
+ https://bugs.webkit.org/show_bug.cgi?id=168405
+
+ Reviewed by Michael Catanzaro.
+
+ Merged Install.md and ReadMe.md into one file.
+
+ * Install.md: Removed.
+ * ReadMe.md: Merged Install.md at the top and updated the rest of the content.
+
</ins><span class="cx"> 2017-01-24 Ryosuke Niwa <rniwa@webkit.org>
</span><span class="cx">
</span><span class="cx"> Modernize editable-text component and add tests
</span></span></pre></div>
<a id="trunkWebsitesperfwebkitorgInstallmd"></a>
<div class="delfile"><h4>Deleted: trunk/Websites/perf.webkit.org/Install.md (212420 => 212421)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/Install.md 2017-02-16 04:25:22 UTC (rev 212420)
+++ trunk/Websites/perf.webkit.org/Install.md 2017-02-16 05:00:25 UTC (rev 212421)
</span><span class="lines">@@ -1,124 +0,0 @@
</span><del>-# Checking Out the Code and Installing Required Applications
-
-The instructions assume you're using Mac OS X as the host server and installing this application at `/Volumes/Data/perf.webkit.org`.
-
-You can choose between using Server.app or install the required tools separately
-
-1. Install Server.app (if you don't want to use Server.app, install PostgreSQL: http://www.postgresql.org/download/macosx/)
-2. Install node.
-3. Install Xcode with command line tools (only needed for svn)
-4. `svn co https://svn.webkit.org/repository/webkit/trunk/Websites/perf.webkit.org /Volumes/Data/perf.webkit.org`
-5. Inside `/Volumes/Data/perf.webkit.org`, run `npm install pg` and `mkdir -m 755 public/data/`
-
-# Testing Local UI Changes with Production Data
-
-The front end has the capability to pull data from a production server without replicating the database locally on OS X (Yosemite and later).
-To use this feature, modify `config.json`'s `remoteServer` entry so that "remoteServer.url" points to your production server,
-and "remoteServer.basicAuth" specifies the username and the password that is used by the production sever.
-
-Remove "basicAuth" entry for production servers that doesn't require a basic authentication (e.g. perf.webkit.org).
-
-```json
-{
- "url": "http://perf.webkit.org",
- "basicAuth": {
- "username": "webkitten",
- "password": "webkitten's secret password"
- }
-}
-```
-
-Then run `tools/remote-cache-server.py start`. This launches a httpd server on port 8080.
-
-The script caches remote server's responses under `public/data/remote-cache` and never revalidates them (to allow offline work).
-If you needed the latest content, delete caches stored in this directory by running `tools/remote-cache-server.py reset`.
-
-
-# Configuring Apache
-
-## Instructions if you're using Server.app
-
- - Enable PHP web applications
- - Go to Server Website / Store Site Files In, change it to `/Volumes/Data/perf.webkit.org/public/`
- - Go to Server Website / Edit advanced settings, enable Allow overrides using .htaccess files
- - httpd config file is located at `/Library/Server/Web/Config/apache2/sites/0000_any_80.conf` (and/or 0000_any_`PORT#`.conf)
-
-## Instructions if you're not using Server.app
-
- - Edit /private/etc/apache2/httpd.conf
-
- 1. Change DocumentRoot to `/Volumes/Data/perf.webkit.org/public/`
- 2. Uncomment `LoadModule php5_module libexec/apache2/libphp5.so`
- 3. Uncomment `LoadModule rewrite_module libexec/apache2/mod_rewrite.so`
- 4. Uncomment `LoadModule deflate_module libexec/apache2/mod_deflate.so`
-
- - In Mavericks and later, copy php.ini to load pdo_pgsql.so pgsql.so.
- `sudo cp /Applications/Server.app/Contents/ServerRoot/etc/php.ini /etc/`
- - In El Capitan and later, comment out the `LockFile` directive in `/private/etc/apache2/extra/httpd-mpm.conf`
- since the directive has been superseded by `Mutex` directive.
-
-## Starting Apache
-
-You can use apachectl to start/stop/restart apache server from the command line:
-
-- Starting httpd: `sudo apachectl start`
-- Stopping httpd: `sudo apachectl stop`
-- Restarting httpd: `sudo apachectl restart`
-
-The apache logs are located at `/private/var/log/apache2`.
-
-## Production Configurations
-
- 1. Update ServerAdmin to your email address
- 2. Add the following directives to enable gzip:
-
- <IfModule mod_deflate.c>
- AddOutputFilterByType DEFLATE text/html text/xml text/javascript application/javascript text/plain application/json application/xml application/xhtml+xml
- </IfModule>
-
- 3. Add the following directives to enable zlib compression and MultiViews on DocumentRoot (perf.webkit.org/public):
-
- Options Indexes MultiViews
- php_flag zlib.output_compression on
-
-### Protecting the Administrative Pages to Prevent Execution of Arbitrary Code
-
-By default, the application gives the administrative privilege to everyone. Anyone can add, remove, or edit tests,
-builders, and other entities in the database.
-
-We recommend protection via Digest Auth on https connection.
-
-Generate a password file via `htdigest -c <path> <realm> <username>`, and then create admin/.htaccess with the following directives
-where `<Realm>` is replaced with the realm of your choice, which will be displayed on the username/password input box:
-
-```
-AuthType Digest
-AuthName "<Realm>"
-AuthDigestProvider file
-AuthUserFile "<Realm>"
-Require valid-user
-```
-
-# Configuring PostgreSQL
-
-Run the following command to setup a Postgres server at `/Volumes/Data/perf.webkit.org/PostgresSQL` (or wherever you'd prefer):
-`python ./tools/setup-database.py /Volumes/Data/perf.webkit.org/PostgresSQL`
-
-It automatically retrieves the database name, the username, and the password from `config.json`.
-
-## Starting PostgreSQL
-
-The setup script automatically starts the database but you may need to run the following command to manually start the database after reboot.
-
-- Starting the database: `/Applications/Server.app/Contents/ServerRoot/usr/bin/pg_ctl -D /Volumes/Data/perf.webkit.org/PostgresSQL -l /Volumes/Data/perf.webkit.org/PostgresSQL/logfile -o "-k /Volumes/Data/perf.webkit.org/PostgresSQL" start`
-- Stopping the database: `/Applications/Server.app/Contents/ServerRoot/usr/bin/pg_ctl -D /Volumes/Data/perf.webkit.org/PostgresSQL -l /Volumes/Data/perf.webkit.org/PostgresSQL/logfile -o "-k /Volumes/Data/perf.webkit.org/PostgresSQL" stop`
-
-## Initializing the Database
-
-Run `database/init-database.sql` in psql as `webkit-perf-db-user`:
-`/Applications/Server.app/Contents/ServerRoot/usr/bin/psql webkit-perf-db -h localhost --username webkit-perf-db-user -f init-database.sql`
-
-## Making a Backup and Restoring
-
-- Backing up the database: `/Applications/Server.app/Contents/ServerRoot/usr/bin/pg_dump -h localhost --no-owner -f <filepath> webkit-perf-db | gzip > backup.gz`
-- Restoring the database: `gunzip -c backup.gz | /Applications/Server.app/Contents/ServerRoot/usr/bin/psql webkit-perf-db -h localhost --username webkit-perf-db-user`
</del></span></pre></div>
<a id="trunkWebsitesperfwebkitorgReadMemd"></a>
<div class="modfile"><h4>Modified: trunk/Websites/perf.webkit.org/ReadMe.md (212420 => 212421)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Websites/perf.webkit.org/ReadMe.md 2017-02-16 04:25:22 UTC (rev 212420)
+++ trunk/Websites/perf.webkit.org/ReadMe.md 2017-02-16 05:00:25 UTC (rev 212421)
</span><span class="lines">@@ -1,117 +1,224 @@
</span><del>-# Concepts
</del><ins>+# WebKit Performance Dashboard
</ins><span class="cx">
</span><del>-## Platform
</del><ins>+The WebKit performane dashboard is a website to track various performance metrics of WebKit: https://perf.webkit.org/
</ins><span class="cx">
</span><del>-A platform is an environmental configuration under which performance tests run. This is typically
-an operating system such as Lion, Mountain Lion, or Windows 7.
</del><ins>+## Checking Out the Code and Installing Required Applications
</ins><span class="cx">
</span><del>-## Builder
</del><ins>+The instructions assume you're using macOS as the host server and installing this application at `/Volumes/Data/perf.webkit.org`.
</ins><span class="cx">
</span><del>-A builder is a physical machine that submits a result of one or more tests to one or more platforms.
-Each builder should have a password it uses to submit the results to this application, and it may also
-have a URL associated with it.
</del><ins>+You can choose between using Server.app or install the required tools separately.
</ins><span class="cx">
</span><del>-## Build
</del><ins>+1. Install Server.app (if you don't want to use Server.app, install PostgreSQL: http://www.postgresql.org/download/macosx/)
+2. Install node.
+3. Install Xcode with command line tools (only needed for svn)
+4. `svn co https://svn.webkit.org/repository/webkit/trunk/Websites/perf.webkit.org /Volumes/Data/perf.webkit.org`
+5. Inside `/Volumes/Data/perf.webkit.org`, run `npm install pg` and `mkdir -m 755 public/data/`
</ins><span class="cx">
</span><del>-A build is a single run of tests on a given builder. It's possible for a single build to have ran multiple
-tests on multiple platforms.
</del><ins>+## Testing Local UI Changes with Production Data
</ins><span class="cx">
</span><del>-## Test Metric
</del><ins>+The front end has the capability to pull data from a production server without replicating the database locally on OS X Yosemite and later.
+To use this feature, modify `config.json`'s `remoteServer` entry so that "remoteServer.url" points to your production server,
+and "remoteServer.basicAuth" specifies the username and the password that is used by the production sever.
</ins><span class="cx">
</span><del>-A test metric is a type of measurement a test makes. A single test may measure multiple metrics such as
-Time (ms), Malloc (bytes), and JSHeap (bytes). The mapping from metrics to units is a function
-(in mathematical sense).
</del><ins>+Remove "basicAuth" entry for production servers that doesn't require a basic authentication (e.g. perf.webkit.org).
</ins><span class="cx">
</span><del>-## Test Configuration
</del><ins>+```json
+{
+ "url": "http://perf.webkit.org",
+ "basicAuth": {
+ "username": "webkitten",
+ "password": "webkitten's secret password"
+ }
+}
+```
</ins><span class="cx">
</span><del>-A test configuration is a combination of a test metric, a platform, and a configuration type: "current",
-"baseline", or "target". With metric, configuration creates a three-level tree structure under a test as follows:
</del><ins>+Then run `tools/remote-cache-server.py start`. This launches a httpd server on port 8080.
</ins><span class="cx">
</span><del>-- MyTest (Test 1)
- - Time (Metric 1)
- - Lion : current (Configuration 1)
- - Lion : baseline (Configuration 2)
- - Lion : target (Configuration 3)
- - Mountain Lion : current (Configuration 4)
- - Mountain Lion : baseline (Configuration 5)
- - Mountain Lion : target (Configuration 6)
- - Malloc (Metric 2)
- - Lion : current (Configuration 7)
- - Lion : baseline (Configuration 8)
- - Lion : target (Configuration 9)
- - Mountain Lion : current (Configuration 10)
- - Mountain Lion : baseline (Configuration 11)
- - Mountain Lion : target (Configuration 12)
-- AnotherTest (Test 2)
- - Time (Metric 3)
- - Lion : current (Configuration 13)
- - Mountain Lion : current (Configuration 14)
</del><ins>+The script caches remote server's responses under `public/data/remote-cache` and never revalidates them (to allow offline work).
+If you needed the latest content, delete caches stored in this directory by running `tools/remote-cache-server.py reset`.
</ins><span class="cx">
</span><del>-## Run and Iteration
</del><ins>+## Configuring Apache
</ins><span class="cx">
</span><del>-A run is a ordered list of values obtained for a single configuration on a single build. For example, a Lion
-builder may execute MyTest 10 times, i.e. 10 iterations, and create a single run after computing the arithmetic
-mean of 10 values obtained in this process. Each run has associated iterations, which represents an individual
-measurement of the same configuration (of a single test metric) in the run.
</del><ins>+### Instructions if you're using Server.app
</ins><span class="cx">
</span><del>-## Aggregation and Aggregator
</del><ins>+ - Enable PHP web applications
+ - Go to Server Website / Store Site Files In, change it to `/Volumes/Data/perf.webkit.org/public/`
+ - Go to Server Website / Edit advanced settings, enable Allow overrides using .htaccess files
+ - httpd config file is located at `/Library/Server/Web/Config/apache2/sites/0000_any_80.conf` (and/or 0000_any_`PORT#`.conf)
</ins><span class="cx">
</span><del>-Aggregation is a process by which a test with child tests synthetically generates results for itself using
-results of sub tests. For example, we may have a page loading test (PageLoadingTest), which loads
-www.webkit.org and www.mozilla.org as follows:
</del><ins>+### Instructions if you're not using Server.app
</ins><span class="cx">
</span><del>-- PageLoadingTest (Test 1)
- - www.webkit.org (Test 2)
- - www.mozilla.org (Test 3)
</del><ins>+ - Edit /private/etc/apache2/httpd.conf
</ins><span class="cx">
</span><del>-(Note that PageLoadingTest, www.webkit.org, and www.mozilla.org each has its own metrics and configurations,
-which are not shown here.)
</del><ins>+ 1. Change DocumentRoot to `/Volumes/Data/perf.webkit.org/public/`
+ 2. Uncomment `LoadModule php5_module libexec/apache2/libphp5.so`
+ 3. Uncomment `LoadModule rewrite_module libexec/apache2/mod_rewrite.so`
+ 4. Uncomment `LoadModule deflate_module libexec/apache2/mod_deflate.so`
</ins><span class="cx">
</span><del>-Then results for a metric, e.g. Time, of PageLoadingTest could be generated from results of the same metric in
-subtests, namely www.webkit.org and www.mozilla.org. The process is called "aggregation", and the exact nature of
-the aggregation is defined in terms of an aggregator. All aggregators are written in JavaScript.
</del><ins>+ - In Mavericks and later, copy php.ini to load pdo_pgsql.so pgsql.so.
+ `sudo cp /Applications/Server.app/Contents/ServerRoot/etc/php.ini /etc/`
+ - In El Capitan and later, you may need to comment out the `LockFile` directive in `/private/etc/apache2/extra/httpd-mpm.conf`
+ as the directive has been superseded by `Mutex` directive.
</ins><span class="cx">
</span><del>-The aggregator for arithmetic mean could be implemented as:
-
- values.reduce(function (a, b) { return a + b; }) / values.length;
</del><ins>+### Starting Apache
</ins><span class="cx">
</span><del>-When a builder reports a result JSON to the application, the background process automatically schedules a job
-to aggregate results for all tests specified in the JSON. The aggregation can also be triggered manually on
-`/admin/tests`.
</del><ins>+You can use apachectl to start/stop/restart apache server from the command line:
</ins><span class="cx">
</span><del>-Reporting Results
-=================
</del><ins>+- Starting httpd: `sudo apachectl start`
+- Stopping httpd: `sudo apachectl stop`
+- Restarting httpd: `sudo apachectl restart`
</ins><span class="cx">
</span><del>-To submit the results of a new test to an instance of the app, you need the following:
</del><ins>+The apache logs are located at `/private/var/log/apache2`.
</ins><span class="cx">
</span><del>- - A slave on `/admin/slaves`
- - A script that submits a JSON payload of the supported format via a HTTP/HTTPS request to `/api/report`
</del><ins>+### Production Configurations
</ins><span class="cx">
</span><del>-JSON Format
------------
</del><ins>+ 1. Update ServerAdmin to your email address
+ 2. Add the following directives to enable gzip:
</ins><span class="cx">
</span><del>-The JSON submitted to `/api/report` should be an array of dictionaries, each of which should
-contain the following key-value pairs representing a single run of tests on a single build:
</del><ins>+ <IfModule mod_deflate.c>
+ AddOutputFilterByType DEFLATE text/html text/xml text/javascript application/javascript text/plain application/json application/xml application/xhtml+xml
+ </IfModule>
</ins><span class="cx">
</span><ins>+ 3. Add the following directives to enable zlib compression and MultiViews on DocumentRoot (perf.webkit.org/public):
+
+ Options Indexes MultiViews
+ php_flag zlib.output_compression on
+
+### Protecting the Administrative Pages to Prevent Execution of Arbitrary Code
+
+By default, the application gives the administrative privilege to everyone. Anyone can add, remove, or edit tests,
+builders, and other entities in the database.
+
+We recommend protecting the administrative pages via Digest Auth on https connection.
+
+Generate a password file via `htdigest -c <path> <realm> <username>`, and then create admin/.htaccess with the following directives
+where `<Realm>` is replaced with the realm of your choice, which will be displayed on the username/password input box:
+
+```
+AuthType Digest
+AuthName "<Realm>"
+AuthDigestProvider file
+AuthUserFile "<Realm>"
+Require valid-user
+```
+
+## Configuring PostgreSQL
+
+Run the following command to setup a Postgres server at `/Volumes/Data/perf.webkit.org/PostgresSQL` (or wherever you'd prefer):
+`python ./tools/setup-database.py /Volumes/Data/perf.webkit.org/PostgresSQL`
+
+It automatically retrieves the database name, the username, and the password from `config.json`.
+
+### Starting PostgreSQL
+
+The setup script automatically starts the database but you may need to run the following command to manually start the database after reboot.
+
+- Starting the database: `/Applications/Server.app/Contents/ServerRoot/usr/bin/pg_ctl -D /Volumes/Data/perf.webkit.org/PostgresSQL -l /Volumes/Data/perf.webkit.org/PostgresSQL/logfile -o "-k /Volumes/Data/perf.webkit.org/PostgresSQL" start`
+- Stopping the database: `/Applications/Server.app/Contents/ServerRoot/usr/bin/pg_ctl -D /Volumes/Data/perf.webkit.org/PostgresSQL -l /Volumes/Data/perf.webkit.org/PostgresSQL/logfile -o "-k /Volumes/Data/perf.webkit.org/PostgresSQL" stop`
+
+### Initializing the Database
+
+Run `database/init-database.sql` in psql as `webkit-perf-db-user`:
+`/Applications/Server.app/Contents/ServerRoot/usr/bin/psql webkit-perf-db -h localhost --username webkit-perf-db-user -f init-database.sql`
+
+### Making a Backup and Restoring
+
+- Backing up the database: `/Applications/Server.app/Contents/ServerRoot/usr/bin/pg_dump -h localhost --no-owner -f <filepath> webkit-perf-db | gzip > backup.gz`
+- Restoring the database: `gunzip -c backup.gz | /Applications/Server.app/Contents/ServerRoot/usr/bin/psql webkit-perf-db -h localhost --username webkit-perf-db-user`
+
+## Concepts
+
+ - **Test** - A test is a collection of metrics such as frame rate and malloced bytes. It can have a children and a parent and form a tree of tests.
+ - **Metric** - A specific metric of a test such as frame rate, runs per second, and time. The list of supported metrics are:
+ - *FrameRate* - Frame rate per second
+ - *Runs* - Runs per second
+ - *Time* and *Duration* - Duration measured in milliseconds
+ - *Size* - Bytes
+ - *Score* - Unit-less scores for benchmarks. Unit is pt or points.
+
+ Metrics can also be an aggreagte of child tests' metrics of the same name. For example, Speedometer benchmark's Time metric is the sum total of subtests' time. The list of supported aggregation types are:
+ - *Arithmetic* - [The arithmetic mean](https://en.wikipedia.org/wiki/Arithmetic_mean)
+ - *Geometric* - [The geometric mean](https://en.wikipedia.org/wiki/Geometric_mean)
+ - *Harmonic* - [The harmonic mean](https://en.wikipedia.org/wiki/Harmonic_mean)
+ - *Total* - [The sum](https://en.wikipedia.org/wiki/Summation)
+
+ - **Platform** - A platform is an environmental configuration under which performance tests run. This is typically an operating system such as Lion, Mountain Lion, or Windows 7, or a combination of an operating system, a particular device, and a configuration: e.g. macOS Sierra MacBookAir7,1.
+ - **Repository** - A repository refers to the name of a collection of software whose verions or revisions need to be associated with a particular data point submitted to the dashboard. For example, WebKit is a repository because each data point submitted to the dashboard has to identify its WebKit revision. Operating systems such as macOS could also be considered as a repository if its version may change over time if an operating system may get updated across data points.
+ - **Commit** - A commit is a specific revision or a version of a repository. e.g. r211196 of WebKit.
+ - **Committer** - A committer is the author of the change in a given repository for a specific commit. e.g. the author of r211196 in WebKit is Ryosuke Niwa.
+ - **Builder** - Like a builder in buildbot, a builder submits data points to the dashboard in terms of a sequence of builds. For example, a builder named "El Capitan MacBookAir7,1" may submit data points for benchmarks ran on MacBookAir7,1 with El Capitan.
+ - (Build) **Slave** - Like a buildlsave in buildbot, a slave is a physical machine that submit data points to the dashboard. e.g. it could be a bot205 which submits data points as either "El Capitan MacBookAir7,1" or "Sierra MacBookAir7,1".
+ - **Build** - A build represents a set of data points reported by a specific builder. e.g. "El Capitan MacBookAir7,1" may report Speedometer score along with its subtests' results. All those data points being to a single build. This is a different concept from a build of software. A single build of WebKit, for example, could be ran on multiple builders (e.g. one MacBookAir and another MacBookPro) to generate two builds in the dashboard.
+ - **Bug Tracker** - A bug or an issue tracking tool such as Bugzilla and Radar.
+ - **Bug** - A bug number associated with a particular bug tracker.
+ - **Analysis Task** - An analysis task is created to analyze a progression or a regression across a range of data points for a specific test metric on a specific platform. Each analysis task can have multiple test groups.
+ - **Test Group** - A specific set of A/B testing tasks. Each group may have multiple build requests for each set A and B.
+ - **Build Request** - A specific testing task in a test group.
+
+See [init-database.sql](init-database.sql) for the complete database schema.
+
+## Data Models for Test Results
+
+In the performance dashboard, each test can have a parent test or arbitrary many child tests or subtests. Each test can then have multiple metrics to measure a specific unit'ed value. For example, Speedometer benchmark has the top level score, which is computed by the total time of running subtests. As such, the top level test has two metrics: *Score* and *Time* which is the aggregated total sum of *Time* metrics of the subtests:
+
+**Speedometer** (A test)
+ + *Score* (A metric of "Speedometer")
+ + *Time : Total* (An aggregated metric of "Speedometer"; total sum of "Time" metrics of all subtests)
+ + **AngularJS-TodoMVC** (A subtest of "Speedometer")
+ - *Time* (A metric of "AngularJS-TodoMVC")
+ + **BackboneJS-TodoMVC** (A subtest of "Speedometer")
+ - *Time* (A metric of "BackboneJS-TodoMVC")
+ + ...
+
+Since each test metric can be measured on arbitrarily platforms (e.g. MacBookAir7,1 on macOS Sierra), the dashboard supports showing the *baseline* results (e.g. benchmark scores on Safari 10) in addition to the results from the *current* sofware (the trunk WebKit build), we use a triple (test metric, platform, type) called a *test configuration* to group a collection of data points reported to the dashboard.
+
+Then each test configuration has arbitrary *test runs*. A test run represents the score or more broadly the result of a single test metric on a specific platform at a particular time. For example, running Speedometer once and reporting the results to the performance dashboard results in the creation of a test run for each of *Score* and *Time : Total* metrics as well as *Time* metrics of all subtests (e.g. AngularJS-TodoMVC). Each test run can then have arbitrarily many *iteration values* which are indivisual measurement of some test metric within the benchmark. For example, Speedometer runs the same test twenty times and uses the average time and the score of those twenty iterations to compute the final score. Each one of twenty iterations constitutes a single iteration value.
+
+Each *test run* are related to one another via *builds* which is uniquely identified by its *builder* and a *build number*. For example in the following sample data points for Speedometer, two test runs 789 and 791 are reported by a build 1001, and test runs 876 and 878 are reported by build 1002. Because Speedometer's total score is computed using the time took to run subtests, a builder reported both values in a single build as expected.
+
+(Speedometer's Score, Sierra MacBookAir7,1, "current")
+ + Test run 789 - Mean: 154, Build: 1001
+ + Iteration 1: 150
+ + Iteration 2: 159
+ + ...
+ + Test run 876 - Mean: 153, Build: 1002
+ + Iteration 1: 152
+ + Iteration 2: 157
+ + ...
+
+(AngularJS-TodoMVC's Time, Sierra MacBookAir7,1, "current")
+ + Test run 791 - Mean: 373, Build: 1001
+ + Iteration 1: 450
+ + Iteration 2: 380
+ + ...
+ + Test run 878 - Mean: 380, Build: 1002
+ + Iteration 1: 448
+ + Iteration 2: 395
+ + ...
+
+## Reporting Results
+
+To submit a new *build*, or a set of data points to an instance of the performance dashboard, you need to do the following:
+
+ - Add a slave on `/admin/slaves` if the slave used in the report has not already been added.
+ - Make a HTTP POST request to `/api/report`.
+
+### Format of Results JSON
+
+The JSON submitted to `/api/report` should be an array of dictionaries, and each dictionary should must contain the following key-value pairs representing a single run of tests and its subtests on a single build:
+
</ins><span class="cx"> - `builderName` - The name of a builder. A single slave may submit to multiple builders.
</span><span class="cx"> - `slaveName` - The name of a slave present on `/admin/slaves`.
</span><span class="cx"> - `slavePassword` - The password associated with the slave.
</span><span class="cx"> - `buildNumber` - The string that uniquely identifies a given build on the builder.
</span><del>-- `buildTime` - The time at which this build started in **UTC** (Use ISO time format such as
- 2013-01-31T22:22:12.121051). This is completely independent of timestamp of repository revisions.
</del><ins>+- `buildTime` - The time at which this build started in **UTC** (Use ISO time format such as `2013-01-31T22:22:12.121051`). This is completely independent of timestamp of repository revisions.
</ins><span class="cx"> - `platform` - The human-readable name of a platform such as `Mountain Lion` or `Windows 7`.
</span><del>-- `revisions` - A dictionary that maps a repository name to a dictionary with "revision" and optionally
- "timestamp" as keys each of which maps to, respectively, the revision in **string** associated with
- the build and the times at which the revision was committed to the repository respectively.
- e.g. `{"WebKit": {"revision": "123", "timestamp": "2001-09-10T17:53:19.000000Z"}}`
</del><ins>+- `revisions` - A dictionary that maps a repository name to a dictionary with "revision" and optionally "timestamp" as keys each of which maps to, respectively, the revision in **string** associated with the build and the times at which the revision was committed to the repository respectively. e.g. `{"WebKit": {"revision": "123", "timestamp": "2001-09-10T17:53:19.000000Z"}}`
</ins><span class="cx"> - `tests` - A dictionary that maps a test name to a dictionary that represents a test. The value of a test
</span><span class="cx"> itself is a dictionary with the following keys:
</span><del>- - `metrics` - A dictionary that maps a metric name to a dictionary of configuration types to an array of
- iteration values. e.g. `{"Time": {"current": [629.1, 654.8, 598.9], "target": [544, 585.1, 556]}}`
- When a metric represents an aggregated value, it should be an array of aggregator names instead. e.g.
- `{"Time": ["Arithmetic", "Geometric"]}` **This format may change in near future**.
- - `url` - The URL of the test. This value should not change over time as only the latest value is stored
- in the application.
</del><ins>+ - `metrics` - A dictionary that maps a metric name to a dictionary of configuration types to an array of iteration values. e.g. `{"Time": {"current": [629.1, 654.8, 598.9], "target": [544, 585.1, 556]}}`
+ When a metric represents an aggregated value, it should be an array of aggregator names instead. e.g. `{"Time": ["Arithmetic", "Geometric"]}` **This format may change in near future**.
+ - `url` - The URL of the test. This value should not change over time as only the latest value is stored in the application.
</ins><span class="cx"> - `tests` - A dictionary of tests; the same format as this dictionary.
</span><span class="cx">
</span><span class="cx"> In the example below, we have the top-level test named "PageLoadTime". It measures two metrics: `Time` and `FrameRate`.
</span><span class="lines">@@ -166,5 +273,3 @@
</span><span class="cx"> }
</span><span class="cx"> }]
</span><span class="cx"> ```
</span><del>-
-FIXME: Add a section describing how the application is structured.
</del></span></pre>
</div>
</div>
</body>
</html>