<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[281919] trunk/Tools</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/281919">281919</a></dd>
<dt>Author</dt> <dd>clopez@igalia.com</dd>
<dt>Date</dt> <dd>2021-09-02 04:45:38 -0700 (Thu, 02 Sep 2021)</dd>
</dl>
<h3>Log Message</h3>
<pre>[GTK][WPE] Port API test runner to python3
https://bugs.webkit.org/show_bug.cgi?id=229782
Reviewed by Philippe Normand.
Port the GTK/WPE API test runner to be compatible with python3
and set it to run with python3 by default.
The runner is still compatible with python2 after this patch, so
we can land this without immediately requiring a restart of the CI.
* Scripts/run-gtk-tests:
(GtkTestRunner): Deleted.
(GtkTestRunner.__init__): Deleted.
(GtkTestRunner._ensure_accessibility_service_is_running): Deleted.
(GtkTestRunner._setup_testing_environment): Deleted.
(GtkTestRunner._tear_down_testing_environment): Deleted.
(GtkTestRunner.is_glib_test): Deleted.
(GtkTestRunner.is_google_test): Deleted.
(GtkTestRunner.is_qt_test): Deleted.
* Scripts/run-perf-tests:
* Scripts/run-wpe-tests:
(WPETestRunner): Deleted.
(WPETestRunner.__init__): Deleted.
(WPETestRunner.is_glib_test): Deleted.
(WPETestRunner.is_google_test): Deleted.
(WPETestRunner.is_qt_test): Deleted.
* glib/api_test_runner.py:
(TestRunner._get_tests_from_google_test_suite):
(TestRunner.run_tests):
(TestRunner.run_tests.number_of_tests):
* glib/common.py:
(parse_output_lines):
* glib/glib_test_runner.py:
(Message.create.read_string):
(Message):
(GLibTestRunner._read_from_pipe):
(GLibTestRunner._read_from_stderr):
(GLibTestRunner.run):</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkToolsCISupportbuildwebkitorgstepspy">trunk/Tools/CISupport/build-webkit-org/steps.py</a></li>
<li><a href="#trunkToolsCISupportewsbuildstepspy">trunk/Tools/CISupport/ews-build/steps.py</a></li>
<li><a href="#trunkToolsCISupportewsbuildsteps_unittestpy">trunk/Tools/CISupport/ews-build/steps_unittest.py</a></li>
<li><a href="#trunkToolsChangeLog">trunk/Tools/ChangeLog</a></li>
<li><a href="#trunkToolsScriptsrungtktests">trunk/Tools/Scripts/run-gtk-tests</a></li>
<li><a href="#trunkToolsScriptsrunperftests">trunk/Tools/Scripts/run-perf-tests</a></li>
<li><a href="#trunkToolsScriptsrunwpetests">trunk/Tools/Scripts/run-wpe-tests</a></li>
<li><a href="#trunkToolsglibapi_test_runnerpy">trunk/Tools/glib/api_test_runner.py</a></li>
<li><a href="#trunkToolsglibcommonpy">trunk/Tools/glib/common.py</a></li>
<li><a href="#trunkToolsglibglib_test_runnerpy">trunk/Tools/glib/glib_test_runner.py</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkToolsCISupportbuildwebkitorgstepspy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/CISupport/build-webkit-org/steps.py (281918 => 281919)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/CISupport/build-webkit-org/steps.py 2021-09-02 11:03:53 UTC (rev 281918)
+++ trunk/Tools/CISupport/build-webkit-org/steps.py 2021-09-02 11:45:38 UTC (rev 281919)
</span><span class="lines">@@ -940,11 +940,11 @@
</span><span class="cx">
</span><span class="cx">
</span><span class="cx"> class RunGtkAPITests(RunGLibAPITests):
</span><del>- command = ["python", "./Tools/Scripts/run-gtk-tests", WithProperties("--%(configuration)s")]
</del><ins>+ command = ["python3", "./Tools/Scripts/run-gtk-tests", WithProperties("--%(configuration)s")]
</ins><span class="cx">
</span><span class="cx">
</span><span class="cx"> class RunWPEAPITests(RunGLibAPITests):
</span><del>- command = ["python", "./Tools/Scripts/run-wpe-tests", WithProperties("--%(configuration)s")]
</del><ins>+ command = ["python3", "./Tools/Scripts/run-wpe-tests", WithProperties("--%(configuration)s")]
</ins><span class="cx">
</span><span class="cx">
</span><span class="cx"> class RunWebDriverTests(shell.Test):
</span></span></pre></div>
<a id="trunkToolsCISupportewsbuildstepspy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/CISupport/ews-build/steps.py (281918 => 281919)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/CISupport/ews-build/steps.py 2021-09-02 11:03:53 UTC (rev 281918)
+++ trunk/Tools/CISupport/ews-build/steps.py 2021-09-02 11:45:38 UTC (rev 281919)
</span><span class="lines">@@ -2857,7 +2857,7 @@
</span><span class="cx"> def start(self):
</span><span class="cx"> platform = self.getProperty('platform')
</span><span class="cx"> if platform == 'gtk':
</span><del>- command = ['python', 'Tools/Scripts/run-gtk-tests',
</del><ins>+ command = ['python3', 'Tools/Scripts/run-gtk-tests',
</ins><span class="cx"> '--{0}'.format(self.getProperty('configuration')),
</span><span class="cx"> '--json-output={0}'.format(self.jsonFileName)]
</span><span class="cx"> self.setCommand(command)
</span></span></pre></div>
<a id="trunkToolsCISupportewsbuildsteps_unittestpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/CISupport/ews-build/steps_unittest.py (281918 => 281919)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/CISupport/ews-build/steps_unittest.py 2021-09-02 11:03:53 UTC (rev 281918)
+++ trunk/Tools/CISupport/ews-build/steps_unittest.py 2021-09-02 11:45:38 UTC (rev 281919)
</span><span class="lines">@@ -3275,7 +3275,7 @@
</span><span class="cx"> self.expectRemoteCommands(
</span><span class="cx"> ExpectShell(workdir='wkdir',
</span><span class="cx"> logEnviron=False,
</span><del>- command=['python', 'Tools/Scripts/run-gtk-tests', '--release', '--json-output={0}'.format(self.jsonFileName)],
</del><ins>+ command=['python3', 'Tools/Scripts/run-gtk-tests', '--release', '--json-output={0}'.format(self.jsonFileName)],
</ins><span class="cx"> logfiles={'json': self.jsonFileName},
</span><span class="cx"> )
</span><span class="cx"> + ExpectShell.log('stdio', stdout='''...
</span><span class="lines">@@ -3552,7 +3552,7 @@
</span><span class="cx"> self.expectRemoteCommands(
</span><span class="cx"> ExpectShell(workdir='wkdir',
</span><span class="cx"> logEnviron=False,
</span><del>- command=['python',
</del><ins>+ command=['python3',
</ins><span class="cx"> 'Tools/Scripts/run-gtk-tests',
</span><span class="cx"> '--debug',
</span><span class="cx"> '--json-output={0}'.format(self.jsonFileName)],
</span></span></pre></div>
<a id="trunkToolsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Tools/ChangeLog (281918 => 281919)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/ChangeLog 2021-09-02 11:03:53 UTC (rev 281918)
+++ trunk/Tools/ChangeLog 2021-09-02 11:45:38 UTC (rev 281919)
</span><span class="lines">@@ -1,3 +1,44 @@
</span><ins>+2021-09-02 Carlos Alberto Lopez Perez <clopez@igalia.com>
+
+ [GTK][WPE] Port API test runner to python3
+ https://bugs.webkit.org/show_bug.cgi?id=229782
+
+ Reviewed by Philippe Normand.
+
+ Port the GTK/WPE API test runner to be compatible with python3
+ and set it to run with python3 by default.
+ The runner is still compatible with python2 after this patch, so
+ we can land this without immediately requiring a restart of the CI.
+
+ * Scripts/run-gtk-tests:
+ (GtkTestRunner): Deleted.
+ (GtkTestRunner.__init__): Deleted.
+ (GtkTestRunner._ensure_accessibility_service_is_running): Deleted.
+ (GtkTestRunner._setup_testing_environment): Deleted.
+ (GtkTestRunner._tear_down_testing_environment): Deleted.
+ (GtkTestRunner.is_glib_test): Deleted.
+ (GtkTestRunner.is_google_test): Deleted.
+ (GtkTestRunner.is_qt_test): Deleted.
+ * Scripts/run-perf-tests:
+ * Scripts/run-wpe-tests:
+ (WPETestRunner): Deleted.
+ (WPETestRunner.__init__): Deleted.
+ (WPETestRunner.is_glib_test): Deleted.
+ (WPETestRunner.is_google_test): Deleted.
+ (WPETestRunner.is_qt_test): Deleted.
+ * glib/api_test_runner.py:
+ (TestRunner._get_tests_from_google_test_suite):
+ (TestRunner.run_tests):
+ (TestRunner.run_tests.number_of_tests):
+ * glib/common.py:
+ (parse_output_lines):
+ * glib/glib_test_runner.py:
+ (Message.create.read_string):
+ (Message):
+ (GLibTestRunner._read_from_pipe):
+ (GLibTestRunner._read_from_stderr):
+ (GLibTestRunner.run):
+
</ins><span class="cx"> 2021-09-01 Jonathan Bedard <jbedard@apple.com>
</span><span class="cx">
</span><span class="cx"> [contributors.json] Relocation (Part 2)
</span></span></pre></div>
<a id="trunkToolsScriptsrungtktests"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/run-gtk-tests (281918 => 281919)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/run-gtk-tests 2021-09-02 11:03:53 UTC (rev 281918)
+++ trunk/Tools/Scripts/run-gtk-tests 2021-09-02 11:45:38 UTC (rev 281919)
</span><span class="lines">@@ -1,6 +1,6 @@
</span><del>-#!/usr/bin/env python
</del><ins>+#!/usr/bin/env python3
</ins><span class="cx"> #
</span><del>-# Copyright (C) 2011, 2012, 2017 Igalia S.L.
</del><ins>+# Copyright (C) 2011, 2012, 2017, 2021 Igalia S.L.
</ins><span class="cx"> #
</span><span class="cx"> # This library is free software; you can redistribute it and/or
</span><span class="cx"> # modify it under the terms of the GNU Library General Public
</span></span></pre></div>
<a id="trunkToolsScriptsrunperftests"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/run-perf-tests (281918 => 281919)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/run-perf-tests 2021-09-02 11:03:53 UTC (rev 281918)
+++ trunk/Tools/Scripts/run-perf-tests 2021-09-02 11:45:38 UTC (rev 281919)
</span><span class="lines">@@ -1,4 +1,4 @@
</span><del>-#!/usr/bin/env python
</del><ins>+#!/usr/bin/env python3
</ins><span class="cx"> # Copyright (C) 2012 Google Inc. All rights reserved.
</span><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span></span></pre></div>
<a id="trunkToolsScriptsrunwpetests"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/run-wpe-tests (281918 => 281919)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/run-wpe-tests 2021-09-02 11:03:53 UTC (rev 281918)
+++ trunk/Tools/Scripts/run-wpe-tests 2021-09-02 11:45:38 UTC (rev 281919)
</span><span class="lines">@@ -1,6 +1,6 @@
</span><del>-#!/usr/bin/env python
</del><ins>+#!/usr/bin/env python3
</ins><span class="cx"> #
</span><del>-# Copyright (C) 2017 Igalia S.L.
</del><ins>+# Copyright (C) 2017, 2021 Igalia S.L.
</ins><span class="cx"> #
</span><span class="cx"> # This library is free software; you can redistribute it and/or
</span><span class="cx"> # modify it under the terms of the GNU Library General Public
</span></span></pre></div>
<a id="trunkToolsglibapi_test_runnerpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/glib/api_test_runner.py (281918 => 281919)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/glib/api_test_runner.py 2021-09-02 11:03:53 UTC (rev 281918)
+++ trunk/Tools/glib/api_test_runner.py 2021-09-02 11:45:38 UTC (rev 281919)
</span><span class="lines">@@ -199,7 +199,7 @@
</span><span class="cx">
</span><span class="cx"> def _get_tests_from_google_test_suite(self, test_program, skipped_test_cases):
</span><span class="cx"> try:
</span><del>- output = subprocess.check_output([test_program, '--gtest_list_tests'], env=self._test_env)
</del><ins>+ output = subprocess.check_output([test_program, '--gtest_list_tests'], env=self._test_env).decode('utf-8')
</ins><span class="cx"> except subprocess.CalledProcessError:
</span><span class="cx"> sys.stderr.write("ERROR: could not list available tests for binary %s.\n" % (test_program))
</span><span class="cx"> sys.stderr.flush()
</span><span class="lines">@@ -313,7 +313,7 @@
</span><span class="cx"> if number_of_executed_subtests_for_test > 1:
</span><span class="cx"> number_of_executed_tests += number_of_executed_subtests_for_test
</span><span class="cx"> number_of_total_tests += number_of_executed_subtests_for_test
</span><del>- for test_case, result in results.iteritems():
</del><ins>+ for test_case, result in results.items():
</ins><span class="cx"> if result in self._expectations.get_expectation(os.path.basename(test), test_case):
</span><span class="cx"> continue
</span><span class="cx">
</span><span class="lines">@@ -329,7 +329,7 @@
</span><span class="cx"> self._tear_down_testing_environment()
</span><span class="cx">
</span><span class="cx"> def number_of_tests(tests):
</span><del>- return sum(len(value) for value in tests.itervalues())
</del><ins>+ return sum(len(value) for value in tests.values())
</ins><span class="cx">
</span><span class="cx"> def report(tests, title, base_dir):
</span><span class="cx"> if not tests:
</span></span></pre></div>
<a id="trunkToolsglibcommonpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/glib/common.py (281918 => 281919)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/glib/common.py 2021-09-02 11:03:53 UTC (rev 281918)
+++ trunk/Tools/glib/common.py 2021-09-02 11:45:38 UTC (rev 281919)
</span><span class="lines">@@ -128,10 +128,12 @@
</span><span class="cx"> chunk = ''
</span><span class="cx"> else:
</span><span class="cx"> raise e
</span><del>- if not chunk:
</del><ins>+
+ if chunk:
+ output += chunk.decode('utf-8')
+ else:
</ins><span class="cx"> read_set.remove(fd)
</span><span class="cx">
</span><del>- output += chunk
</del><span class="cx"> while '\n' in output:
</span><span class="cx"> pos = output.find('\n')
</span><span class="cx"> parse_line_callback(output[:pos + 1])
</span></span></pre></div>
<a id="trunkToolsglibglib_test_runnerpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/glib/glib_test_runner.py (281918 => 281919)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/glib/glib_test_runner.py 2021-09-02 11:03:53 UTC (rev 281918)
+++ trunk/Tools/glib/glib_test_runner.py 2021-09-02 11:45:38 UTC (rev 281919)
</span><span class="lines">@@ -37,12 +37,12 @@
</span><span class="cx"> LOG_MAX_RESULT,
</span><span class="cx"> LOG_MESSAGE,
</span><span class="cx"> LOG_START_SUITE,
</span><del>- LOG_STOP_SUITE) = range(12)
</del><ins>+ LOG_STOP_SUITE) = list(range(12))
</ins><span class="cx">
</span><span class="cx"> (TEST_RUN_SUCCESS,
</span><span class="cx"> TEST_RUN_SKIPPED,
</span><span class="cx"> TEST_RUN_FAILURE,
</span><del>- TEST_RUN_INCOMPLETE) = range(4)
</del><ins>+ TEST_RUN_INCOMPLETE) = list(range(4))
</ins><span class="cx">
</span><span class="cx">
</span><span class="cx"> class TestTimeout(Exception):
</span><span class="lines">@@ -72,7 +72,7 @@
</span><span class="cx"> values = []
</span><span class="cx"> for i in range(n):
</span><span class="cx"> str_len = read_unsigned(bytes)[0]
</span><del>- values.append(struct.unpack('%ds' % str_len, bytes.read(str_len))[0])
</del><ins>+ values.append(struct.unpack('%ds' % str_len, bytes.read(str_len))[0].decode('utf-8'))
</ins><span class="cx"> return values
</span><span class="cx">
</span><span class="cx"> bytes = BytesIO(data)
</span><span class="lines">@@ -127,7 +127,7 @@
</span><span class="cx"> self._subtest_message([message.strings[0]])
</span><span class="cx">
</span><span class="cx"> def _read_from_pipe(self, pipe_r):
</span><del>- data = ''
</del><ins>+ data = b''
</ins><span class="cx"> read_set = [pipe_r]
</span><span class="cx"> while read_set:
</span><span class="cx"> try:
</span><span class="lines">@@ -165,7 +165,7 @@
</span><span class="cx"> if not buffer:
</span><span class="cx"> return data
</span><span class="cx">
</span><del>- data += buffer
</del><ins>+ data += buffer.decode('utf-8')
</ins><span class="cx">
</span><span class="cx"> @staticmethod
</span><span class="cx"> def _start_timeout(timeout):
</span><span class="lines">@@ -248,7 +248,10 @@
</span><span class="cx"> if not self._results:
</span><span class="cx"> sys.stdout.write('TEST: %s...\n' % self._test_binary)
</span><span class="cx"> sys.stdout.flush()
</span><del>- p = subprocess.Popen(command, stderr=subprocess.PIPE, env=env)
</del><ins>+ if sys.version_info.major > 2:
+ p = subprocess.Popen(command, stderr=subprocess.PIPE, env=env, pass_fds=[pipe_w])
+ else:
+ p = subprocess.Popen(command, stderr=subprocess.PIPE, env=env)
</ins><span class="cx"> self._stderr_fd = p.stderr.fileno()
</span><span class="cx"> os.close(pipe_w)
</span><span class="cx">
</span></span></pre>
</div>
</div>
</body>
</html>