<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[234750] trunk/PerformanceTests</title>
</head>
<body>

<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt;  }
#msg dl a { font-weight: bold}
#msg dl a:link    { color:#fc3; }
#msg dl a:active  { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff  {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/234750">234750</a></dd>
<dt>Author</dt> <dd>commit-queue@webkit.org</dd>
<dt>Date</dt> <dd>2018-08-09 21:53:42 -0700 (Thu, 09 Aug 2018)</dd>
</dl>

<h3>Log Message</h3>
<pre>Add ability to ignore process prewarming for launch time benchmark
https://bugs.webkit.org/show_bug.cgi?id=188462

Patch by Ben Richards <benton_richards@apple.com> on 2018-08-09
Reviewed by Ryosuke Niwa.

Added a flag to the new tab benchmark that will open a blank tab before the tab to be measured in order to ignore process prewarming.

* LaunchTime/launch_time.py:
(DefaultLaunchTimeHandler):
(DefaultLaunchTimeHandler.on_receive_stop_time): Deleted.
(DefaultLaunchTimeHandler.on_receive_stop_signal):
(DefaultLaunchTimeHandler.do_HEAD):
(DefaultLaunchTimeHandler.do_GET):
(DefaultLaunchTimeHandler.do_POST):
(LaunchTimeBenchmark):
(LaunchTimeBenchmark._standard_deviation): Fixed divide by zero bug when '-n' is set to 1
(LaunchTimeBenchmark.open_tab): Added option to open a blank tab
(LaunchTimeBenchmark.run):
* LaunchTime/new_tab.py:
(NewTabBenchmark.initialize):
(NewTabBenchmark.run_iteration):
(NewTabBenchmark.will_parse_arguments):
(NewTabBenchmark.did_parse_arguments):
(NewTabBenchmark.ResponseHandler.Handler.get_test_page):
(NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted.
(NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_signal):
(NewTabBenchmark):
* LaunchTime/startup.py:
(StartupBenchmark.ResponseHandler.Handler.get_test_page):
(StartupBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted.
(StartupBenchmark.ResponseHandler.Handler.on_receive_stop_signal):
(StartupBenchmark):</pre>

<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkPerformanceTestsChangeLog">trunk/PerformanceTests/ChangeLog</a></li>
<li><a href="#trunkPerformanceTestsLaunchTimelaunch_timepy">trunk/PerformanceTests/LaunchTime/launch_time.py</a></li>
<li><a href="#trunkPerformanceTestsLaunchTimenew_tabpy">trunk/PerformanceTests/LaunchTime/new_tab.py</a></li>
<li><a href="#trunkPerformanceTestsLaunchTimestartuppy">trunk/PerformanceTests/LaunchTime/startup.py</a></li>
</ul>

</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkPerformanceTestsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/ChangeLog (234749 => 234750)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/ChangeLog 2018-08-10 02:13:32 UTC (rev 234749)
+++ trunk/PerformanceTests/ChangeLog    2018-08-10 04:53:42 UTC (rev 234750)
</span><span class="lines">@@ -1,3 +1,38 @@
</span><ins>+2018-08-09  Ben Richards  <benton_richards@apple.com>
+
+        Add ability to ignore process prewarming for launch time benchmark
+        https://bugs.webkit.org/show_bug.cgi?id=188462
+
+        Reviewed by Ryosuke Niwa.
+
+        Added a flag to the new tab benchmark that will open a blank tab before the tab to be measured in order to ignore process prewarming.
+
+        * LaunchTime/launch_time.py:
+        (DefaultLaunchTimeHandler):
+        (DefaultLaunchTimeHandler.on_receive_stop_time): Deleted.
+        (DefaultLaunchTimeHandler.on_receive_stop_signal):
+        (DefaultLaunchTimeHandler.do_HEAD):
+        (DefaultLaunchTimeHandler.do_GET):
+        (DefaultLaunchTimeHandler.do_POST):
+        (LaunchTimeBenchmark):
+        (LaunchTimeBenchmark._standard_deviation): Fixed divide by zero bug when '-n' is set to 1
+        (LaunchTimeBenchmark.open_tab): Added option to open a blank tab
+        (LaunchTimeBenchmark.run):
+        * LaunchTime/new_tab.py:
+        (NewTabBenchmark.initialize):
+        (NewTabBenchmark.run_iteration):
+        (NewTabBenchmark.will_parse_arguments):
+        (NewTabBenchmark.did_parse_arguments):
+        (NewTabBenchmark.ResponseHandler.Handler.get_test_page):
+        (NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted.
+        (NewTabBenchmark.ResponseHandler.Handler.on_receive_stop_signal):
+        (NewTabBenchmark):
+        * LaunchTime/startup.py:
+        (StartupBenchmark.ResponseHandler.Handler.get_test_page):
+        (StartupBenchmark.ResponseHandler.Handler.on_receive_stop_time): Deleted.
+        (StartupBenchmark.ResponseHandler.Handler.on_receive_stop_signal):
+        (StartupBenchmark):
+
</ins><span class="cx"> 2018-07-26  Andy VanWagoner  <andy@vanwagoner.family>
</span><span class="cx"> 
</span><span class="cx">         [INTL] Remove INTL sub-feature compile flags
</span></span></pre></div>
<a id="trunkPerformanceTestsLaunchTimelaunch_timepy"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/LaunchTime/launch_time.py (234749 => 234750)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/LaunchTime/launch_time.py 2018-08-10 02:13:32 UTC (rev 234749)
+++ trunk/PerformanceTests/LaunchTime/launch_time.py    2018-08-10 04:53:42 UTC (rev 234750)
</span><span class="lines">@@ -41,21 +41,12 @@
</span><span class="cx">         </html>
</span><span class="cx">         '''
</span><span class="cx"> 
</span><del>-    def get_blank_page(self):
-        return '''<!DOCTYPE html>
-        <html>
-          <head>
-            <title>Launch Time Benchmark</title>
-            <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-          </head>
-        </html>'''
-
-    def on_receive_stop_time(self, time):
</del><ins>+    def on_receive_stop_signal(self, data):
</ins><span class="cx">         pass
</span><span class="cx"> 
</span><span class="cx">     def do_HEAD(self):
</span><span class="cx">         self.send_response(200)
</span><del>-        self.send_header('Content-type', 'text/hetml')
</del><ins>+        self.send_header('Content-type', 'text/html')
</ins><span class="cx">         self.end_headers()
</span><span class="cx"> 
</span><span class="cx">     def do_GET(self):
</span><span class="lines">@@ -62,7 +53,8 @@
</span><span class="cx">         self.send_response(200)
</span><span class="cx">         self.send_header('Content-type', 'text/html')
</span><span class="cx">         self.end_headers()
</span><del>-        self.wfile.write(self.get_blank_page() if self.path == '/blank' else self.get_test_page())
</del><ins>+        if not self.path.startswith('/blank'):
+            self.wfile.write(self.get_test_page())
</ins><span class="cx">         self.wfile.close()
</span><span class="cx"> 
</span><span class="cx">     def do_POST(self):
</span><span class="lines">@@ -73,10 +65,8 @@
</span><span class="cx">         self.wfile.close()
</span><span class="cx"> 
</span><span class="cx">         data_string = self.rfile.read(int(self.headers['Content-Length']))
</span><del>-        time = float(data_string)
</del><ins>+        self.on_receive_stop_signal(data_string)
</ins><span class="cx"> 
</span><del>-        self.on_receive_stop_time(time)
-
</del><span class="cx">     def log_message(self, format, *args):
</span><span class="cx">         pass
</span><span class="cx"> 
</span><span class="lines">@@ -177,7 +167,8 @@
</span><span class="cx">     def _standard_deviation(self, results, mean=None):
</span><span class="cx">         if mean is None:
</span><span class="cx">             mean = sum(results) / float(len(results))
</span><del>-        variance = sum((x - mean) ** 2 for x in results) / float(len(results) - 1)
</del><ins>+        divisor = float(len(results) - 1) if len(results) > 1 else float(len(results))
+        variance = sum((x - mean) ** 2 for x in results) / divisor
</ins><span class="cx">         return sqrt(variance)
</span><span class="cx"> 
</span><span class="cx">     def _compute_results(self, results):
</span><span class="lines">@@ -197,9 +188,13 @@
</span><span class="cx">         for i in range(self.iteration_groups):
</span><span class="cx">             yield self.wait_time_low + increment_per_group * i
</span><span class="cx"> 
</span><del>-    def open_tab(self):
-        call(['open', '-a', self._browser_bundle_path,
-            'http://localhost:{}/{}'.format(self._port, self._open_count)])
</del><ins>+    def open_tab(self, blank=False):
+        if blank:
+            call(['open', '-a', self._browser_bundle_path,
+                'http://localhost:{}/blank/{}'.format(self._port, self._open_count)])
+        else:
+            call(['open', '-a', self._browser_bundle_path,
+                'http://localhost:{}/{}'.format(self._port, self._open_count)])
</ins><span class="cx">         self._open_count += 1
</span><span class="cx"> 
</span><span class="cx">     def launch_browser(self):
</span><span class="lines">@@ -273,8 +268,8 @@
</span><span class="cx">                         results_by_iteration_number[i].append(result_in_ms)
</span><span class="cx">                     except KeyboardInterrupt:
</span><span class="cx">                         raise KeyboardInterrupt
</span><del>-                    except:
-                        self._exit_due_to_exception('(Test {} failed)\n'.format(i + 1 if self._verbose else i))
</del><ins>+                    except Exception as error:
+                        self._exit_due_to_exception('(Test {} failed) {}: {}\n'.format(i + 1 if self._verbose else i, type(error).__name__, error))
</ins><span class="cx">                 if not self._verbose:
</span><span class="cx">                     print ''
</span><span class="cx"> 
</span></span></pre></div>
<a id="trunkPerformanceTestsLaunchTimenew_tabpy"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/LaunchTime/new_tab.py (234749 => 234750)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/LaunchTime/new_tab.py     2018-08-10 02:13:32 UTC (rev 234749)
+++ trunk/PerformanceTests/LaunchTime/new_tab.py        2018-08-10 04:53:42 UTC (rev 234750)
</span><span class="lines">@@ -36,16 +36,22 @@
</span><span class="cx">         self.start_time = None
</span><span class="cx">         self.stop_time = None
</span><span class="cx">         self.stop_signal_was_received = Event()
</span><ins>+        self.allow_prewarm = True
</ins><span class="cx"> 
</span><span class="cx">     def run_iteration(self):
</span><ins>+        tabs_to_open = 1 if self.allow_prewarm else 2
+        self.stop_time = None
+        for _ in range(tabs_to_open - 1):
+            self.open_tab(blank=True)
</ins><span class="cx">         self.start_time = time.time() * 1000
</span><span class="cx">         self.open_tab()
</span><span class="cx">         while self.stop_time is None:
</span><span class="cx">             self.stop_signal_was_received.wait()
</span><span class="cx">         result = self.stop_time - self.start_time
</span><del>-        self.stop_time = None
</del><span class="cx">         self.stop_signal_was_received.clear()
</span><del>-        self.close_tab()
</del><ins>+        for _ in range(tabs_to_open):
+            self.close_tab()
+
</ins><span class="cx">         return result
</span><span class="cx"> 
</span><span class="cx">     def group_init(self):
</span><span class="lines">@@ -56,6 +62,8 @@
</span><span class="cx">             help='number of groups of iterations to run (default: {})'.format(self.iteration_groups))
</span><span class="cx">         self.argument_parser.add_argument('-w', '--wait-time', type=self._parse_wait_time,
</span><span class="cx">             help='wait time to use between iterations or range to scan (format is "N" or "N:M" where N < M, default: {}:{})'.format(self.wait_time_low, self.wait_time_high))
</span><ins>+        self.argument_parser.add_argument('--no-prewarm', action='store_true',
+            help='attempt to ignore process prewarming (will most likely raise standard deviation)')
</ins><span class="cx"> 
</span><span class="cx">     def did_parse_arguments(self, args):
</span><span class="cx">         if args.groups:
</span><span class="lines">@@ -62,6 +70,8 @@
</span><span class="cx">             self.iteration_groups = args.groups
</span><span class="cx">         if args.wait_time:
</span><span class="cx">             self.wait_time_low, self.wait_time_high = args.wait_time
</span><ins>+        if args.no_prewarm:
+            self.allow_prewarm = False
</ins><span class="cx"> 
</span><span class="cx">     @staticmethod
</span><span class="cx">     def ResponseHandler(new_tab_benchmark):
</span><span class="lines">@@ -74,8 +84,8 @@
</span><span class="cx">                     <meta http-equiv="Content-Type" content="text/html" />
</span><span class="cx">                     <script>
</span><span class="cx">                         function sendDone() {
</span><del>-                            var time = performance.timing.navigationStart
-                            var request = new XMLHttpRequest();
</del><ins>+                            const time = performance.timing.navigationStart
+                            const request = new XMLHttpRequest();
</ins><span class="cx">                             request.open("POST", "done", false);
</span><span class="cx">                             request.setRequestHeader('Content-Type', 'application/json');
</span><span class="cx">                             request.send(JSON.stringify(time));
</span><span class="lines">@@ -89,8 +99,8 @@
</span><span class="cx">                 </html>
</span><span class="cx">                 '''
</span><span class="cx"> 
</span><del>-            def on_receive_stop_time(self, stop_time):
-                new_tab_benchmark.stop_time = stop_time
</del><ins>+            def on_receive_stop_signal(self, data):
+                new_tab_benchmark.stop_time = float(data)
</ins><span class="cx">                 new_tab_benchmark.stop_signal_was_received.set()
</span><span class="cx"> 
</span><span class="cx">         return Handler
</span></span></pre></div>
<a id="trunkPerformanceTestsLaunchTimestartuppy"></a>
<div class="modfile"><h4>Modified: trunk/PerformanceTests/LaunchTime/startup.py (234749 => 234750)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/PerformanceTests/LaunchTime/startup.py     2018-08-10 02:13:32 UTC (rev 234749)
+++ trunk/PerformanceTests/LaunchTime/startup.py        2018-08-10 04:53:42 UTC (rev 234750)
</span><span class="lines">@@ -50,8 +50,8 @@
</span><span class="cx">                 </html>
</span><span class="cx">                 '''
</span><span class="cx"> 
</span><del>-            def on_receive_stop_time(self, stop_time):
-                startup_benchmark.stop_time = stop_time
</del><ins>+            def on_receive_stop_signal(self, data):
+                startup_benchmark.stop_time = float(data)
</ins><span class="cx">                 startup_benchmark.stop_signal_was_received.set()
</span><span class="cx"> 
</span><span class="cx">         return Handler
</span></span></pre>
</div>
</div>

</body>
</html>