<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[172214] trunk/Tools</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/172214">172214</a></dd>
<dt>Author</dt> <dd>bjonesbe@adobe.com</dd>
<dt>Date</dt> <dd>2014-08-07 09:37:59 -0700 (Thu, 07 Aug 2014)</dd>
</dl>
<h3>Log Message</h3>
<pre>The support directory shouldn't be skipped unconditionally in test import
https://bugs.webkit.org/show_bug.cgi?id=135660
Reviewed by Ryosuke Niwa.
The 'DIRS_TO_SKIP' should only be skipped when in the root directory
of the test repo, as that's the only time they are special. In
addition, instead of hardcoding .hg and .git as special, skip all
directories that begin with '.', just like with files that begin with '.'.
In order to make this work, the root directory must always be
passed in, so the interface to the script has been changed to take the
root directory, and if one wants to only import a subset of the tests,
a new -t option can be used to limit the tests imported.
* Scripts/webkitpy/w3c/test_importer.py:
(main): Remove repo_dir command line argument.
(parse_args): Add -t option and set expected non-option args to 1.
(TestImporter.__init__): Remove repo_dir.
(TestImporter.do_import): Handle the varying number of import
directories.
(TestImporter.should_keep_subdir): Helper for find_importable_tests to
determine if a subdirectory should be skipped.
(TestImporter.find_importable_tests): Filter directories using new
helper.
(TestImporter.import_tests): Remove use of repo_dir.
(TestImporter.setup_destination_directory): Unused, Deleted.
* Scripts/webkitpy/w3c/test_importer_unittest.py:
(TestImporterTest.test_import_dir_with_no_tests_and_no_hg): Update for
new API.
(TestImporterTest.test_import_dir_with_no_tests): Ditto.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkToolsChangeLog">trunk/Tools/ChangeLog</a></li>
<li><a href="#trunkToolsScriptswebkitpyw3ctest_importerpy">trunk/Tools/Scripts/webkitpy/w3c/test_importer.py</a></li>
<li><a href="#trunkToolsScriptswebkitpyw3ctest_importer_unittestpy">trunk/Tools/Scripts/webkitpy/w3c/test_importer_unittest.py</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkToolsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Tools/ChangeLog (172213 => 172214)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/ChangeLog 2014-08-07 16:23:05 UTC (rev 172213)
+++ trunk/Tools/ChangeLog 2014-08-07 16:37:59 UTC (rev 172214)
</span><span class="lines">@@ -1,3 +1,37 @@
</span><ins>+2014-08-07 Bem Jones-Bey <bjonesbe@adobe.com>
+
+ The support directory shouldn't be skipped unconditionally in test import
+ https://bugs.webkit.org/show_bug.cgi?id=135660
+
+ Reviewed by Ryosuke Niwa.
+
+ The 'DIRS_TO_SKIP' should only be skipped when in the root directory
+ of the test repo, as that's the only time they are special. In
+ addition, instead of hardcoding .hg and .git as special, skip all
+ directories that begin with '.', just like with files that begin with '.'.
+
+ In order to make this work, the root directory must always be
+ passed in, so the interface to the script has been changed to take the
+ root directory, and if one wants to only import a subset of the tests,
+ a new -t option can be used to limit the tests imported.
+
+ * Scripts/webkitpy/w3c/test_importer.py:
+ (main): Remove repo_dir command line argument.
+ (parse_args): Add -t option and set expected non-option args to 1.
+ (TestImporter.__init__): Remove repo_dir.
+ (TestImporter.do_import): Handle the varying number of import
+ directories.
+ (TestImporter.should_keep_subdir): Helper for find_importable_tests to
+ determine if a subdirectory should be skipped.
+ (TestImporter.find_importable_tests): Filter directories using new
+ helper.
+ (TestImporter.import_tests): Remove use of repo_dir.
+ (TestImporter.setup_destination_directory): Unused, Deleted.
+ * Scripts/webkitpy/w3c/test_importer_unittest.py:
+ (TestImporterTest.test_import_dir_with_no_tests_and_no_hg): Update for
+ new API.
+ (TestImporterTest.test_import_dir_with_no_tests): Ditto.
+
</ins><span class="cx"> 2014-08-07 Carlos Garcia Campos <cgarcia@igalia.com>
</span><span class="cx">
</span><span class="cx"> [GTK] Use WebKitNavigationAction also for WebKitNavigationPolicyDecision
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpyw3ctest_importerpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/w3c/test_importer.py (172213 => 172214)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/w3c/test_importer.py 2014-08-07 16:23:05 UTC (rev 172213)
+++ trunk/Tools/Scripts/webkitpy/w3c/test_importer.py 2014-08-07 16:37:59 UTC (rev 172214)
</span><span class="lines">@@ -93,22 +93,13 @@
</span><span class="cx"> def main(_argv, _stdout, _stderr):
</span><span class="cx"> options, args = parse_args()
</span><span class="cx"> import_dir = args[0]
</span><del>- if len(args) == 1:
- repo_dir = import_dir
- else:
- repo_dir = args[1]
</del><span class="cx">
</span><span class="cx"> if not os.path.exists(import_dir):
</span><span class="cx"> sys.exit('Source directory %s not found!' % import_dir)
</span><span class="cx">
</span><del>- if not os.path.exists(repo_dir):
- sys.exit('Repository directory %s not found!' % repo_dir)
- if not repo_dir in import_dir:
- sys.exit('Repository directory %s must be a parent of %s' % (repo_dir, import_dir))
-
</del><span class="cx"> configure_logging()
</span><span class="cx">
</span><del>- test_importer = TestImporter(Host(), import_dir, repo_dir, options)
</del><ins>+ test_importer = TestImporter(Host(), import_dir, options)
</ins><span class="cx"> test_importer.do_import()
</span><span class="cx">
</span><span class="cx">
</span><span class="lines">@@ -129,23 +120,25 @@
</span><span class="cx">
</span><span class="cx">
</span><span class="cx"> def parse_args():
</span><del>- parser = optparse.OptionParser(usage='usage: %prog [options] w3c_test_directory [repo_directory]')
</del><ins>+ parser = optparse.OptionParser(usage='usage: %prog [options] w3c_test_directory')
</ins><span class="cx"> parser.add_option('-n', '--no-overwrite', dest='overwrite', action='store_false', default=True,
</span><span class="cx"> help='Flag to prevent duplicate test files from overwriting existing tests. By default, they will be overwritten')
</span><span class="cx"> parser.add_option('-a', '--all', action='store_true', default=False,
</span><span class="cx"> help='Import all tests including reftests, JS tests, and manual/pixel tests. By default, only reftests and JS tests are imported')
</span><span class="cx"> parser.add_option('-d', '--dest-dir', dest='destination', default='w3c',
</span><span class="cx"> help='Import into a specified directory relative to the LayoutTests root. By default, imports into w3c')
</span><ins>+ parser.add_option('-t', '--test-path', action='append', dest='test_paths', default=[],
+ help='Import only tests in the supplied subdirectory of the w3c_test_directory. Can be supplied multiple times to give multiple paths')
</ins><span class="cx">
</span><span class="cx"> options, args = parser.parse_args()
</span><del>- if len(args) not in (1, 2):
</del><ins>+ if len(args) != 1:
</ins><span class="cx"> parser.error('Incorrect number of arguments')
</span><span class="cx"> return options, args
</span><span class="cx">
</span><span class="cx">
</span><span class="cx"> class TestImporter(object):
</span><span class="cx">
</span><del>- def __init__(self, host, source_directory, repo_dir, options):
</del><ins>+ def __init__(self, host, source_directory, options):
</ins><span class="cx"> self.host = host
</span><span class="cx"> self.source_directory = source_directory
</span><span class="cx"> self.options = options
</span><span class="lines">@@ -154,7 +147,6 @@
</span><span class="cx">
</span><span class="cx"> webkit_finder = WebKitFinder(self.filesystem)
</span><span class="cx"> self._webkit_root = webkit_finder.webkit_base()
</span><del>- self.repo_dir = repo_dir
</del><span class="cx">
</span><span class="cx"> self.destination_directory = webkit_finder.path_from_webkit_base("LayoutTests", options.destination)
</span><span class="cx">
</span><span class="lines">@@ -163,7 +155,11 @@
</span><span class="cx"> self.import_list = []
</span><span class="cx">
</span><span class="cx"> def do_import(self):
</span><del>- self.find_importable_tests(self.source_directory)
</del><ins>+ if len(self.options.test_paths) == 0:
+ self.find_importable_tests(self.source_directory)
+ else:
+ for test_path in self.options.test_paths:
+ self.find_importable_tests(os.path.join(self.source_directory, test_path))
</ins><span class="cx"> self.load_changeset()
</span><span class="cx"> self.import_tests()
</span><span class="cx">
</span><span class="lines">@@ -174,6 +170,11 @@
</span><span class="cx"> except (OSError, ScriptError):
</span><span class="cx"> self.changeset = CHANGESET_NOT_AVAILABLE
</span><span class="cx">
</span><ins>+ def should_keep_subdir(self, root, subdir):
+ DIRS_TO_SKIP = ('work-in-progress', 'tools', 'support')
+ should_skip = subdir.startswith('.') or (root == self.source_directory and subdir in DIRS_TO_SKIP)
+ return not should_skip
+
</ins><span class="cx"> def find_importable_tests(self, directory):
</span><span class="cx"> # FIXME: use filesystem
</span><span class="cx"> for root, dirs, files in os.walk(directory):
</span><span class="lines">@@ -182,10 +183,7 @@
</span><span class="cx"> reftests = 0
</span><span class="cx"> jstests = 0
</span><span class="cx">
</span><del>- DIRS_TO_SKIP = ('.git', '.hg', 'work-in-progress', 'tools', 'support')
- for d in DIRS_TO_SKIP:
- if d in dirs:
- dirs.remove(d)
</del><ins>+ dirs[:] = [subdir for subdir in dirs if self.should_keep_subdir(root, subdir)]
</ins><span class="cx">
</span><span class="cx"> copy_list = []
</span><span class="cx">
</span><span class="lines">@@ -273,7 +271,7 @@
</span><span class="cx">
</span><span class="cx"> orig_path = dir_to_copy['dirname']
</span><span class="cx">
</span><del>- subpath = os.path.relpath(orig_path, self.repo_dir)
</del><ins>+ subpath = os.path.relpath(orig_path, self.source_directory)
</ins><span class="cx"> new_path = os.path.join(self.destination_directory, subpath)
</span><span class="cx">
</span><span class="cx"> if not(os.path.exists(new_path)):
</span><span class="lines">@@ -351,18 +349,6 @@
</span><span class="cx"> for prefixed_property in sorted(total_prefixed_properties, key=lambda p: total_prefixed_properties[p]):
</span><span class="cx"> _log.info(' %s: %s', prefixed_property, total_prefixed_properties[prefixed_property])
</span><span class="cx">
</span><del>- def setup_destination_directory(self):
- """ Creates a destination directory that mirrors that of the source directory """
-
- new_subpath = self.source_directory[len(self.repo_dir):]
-
- destination_directory = os.path.join(self.destination_directory, new_subpath)
-
- if not os.path.exists(destination_directory):
- os.makedirs(destination_directory)
-
- _log.info('Tests will be imported into: %s', destination_directory)
-
</del><span class="cx"> def remove_deleted_files(self, import_directory, new_file_list):
</span><span class="cx"> """ Reads an import log in |import_directory|, compares it to the |new_file_list|, and removes files not in the new list."""
</span><span class="cx">
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpyw3ctest_importer_unittestpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/w3c/test_importer_unittest.py (172213 => 172214)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/w3c/test_importer_unittest.py 2014-08-07 16:23:05 UTC (rev 172213)
+++ trunk/Tools/Scripts/webkitpy/w3c/test_importer_unittest.py 2014-08-07 16:37:59 UTC (rev 172214)
</span><span class="lines">@@ -39,12 +39,12 @@
</span><span class="cx"> from webkitpy.w3c.test_importer import TestImporter
</span><span class="cx">
</span><span class="cx">
</span><del>-FAKE_SOURCE_DIR = '/blink/w3c'
-FAKE_REPO_DIR = '/blink'
</del><ins>+FAKE_SOURCE_DIR = '/tests/csswg'
+FAKE_TEST_PATH = 'css-fake-1'
</ins><span class="cx">
</span><span class="cx"> FAKE_FILES = {
</span><del>- '/blink/w3c/empty_dir/README.txt': '',
- '/mock-checkout/LayoutTests/w3c/README.txt': '',
</del><ins>+ '/tests/csswg/css-fake-1/empty_dir/README.txt': '',
+ '/mock-checkout/LayoutTests/w3c/css-fake-1/README.txt': '',
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> class TestImporterTest(unittest.TestCase):
</span><span class="lines">@@ -54,7 +54,7 @@
</span><span class="cx"> host.executive = MockExecutive2(exception=OSError())
</span><span class="cx"> host.filesystem = MockFileSystem(files=FAKE_FILES)
</span><span class="cx">
</span><del>- importer = TestImporter(host, FAKE_SOURCE_DIR, FAKE_REPO_DIR, optparse.Values({"overwrite": False, 'destination': 'w3c'}))
</del><ins>+ importer = TestImporter(host, FAKE_SOURCE_DIR, optparse.Values({"overwrite": False, 'destination': 'w3c', 'test_paths': [FAKE_TEST_PATH]}))
</ins><span class="cx">
</span><span class="cx"> oc = OutputCapture()
</span><span class="cx"> oc.capture_output()
</span><span class="lines">@@ -68,7 +68,7 @@
</span><span class="cx"> host.executive = MockExecutive2(exception=ScriptError("abort: no repository found in '/Volumes/Source/src/wk/Tools/Scripts/webkitpy/w3c' (.hg not found)!"))
</span><span class="cx"> host.filesystem = MockFileSystem(files=FAKE_FILES)
</span><span class="cx">
</span><del>- importer = TestImporter(host, FAKE_SOURCE_DIR, FAKE_REPO_DIR, optparse.Values({"overwrite": False, 'destination': 'w3c'}))
</del><ins>+ importer = TestImporter(host, FAKE_SOURCE_DIR, optparse.Values({"overwrite": False, 'destination': 'w3c', 'test_paths': [FAKE_TEST_PATH]}))
</ins><span class="cx"> oc = OutputCapture()
</span><span class="cx"> oc.capture_output()
</span><span class="cx"> try:
</span></span></pre>
</div>
</div>
</body>
</html>