<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[212579] trunk/Tools</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/212579">212579</a></dd>
<dt>Author</dt> <dd>commit-queue@webkit.org</dd>
<dt>Date</dt> <dd>2017-02-17 14:41:49 -0800 (Fri, 17 Feb 2017)</dd>
</dl>
<h3>Log Message</h3>
<pre>EWS should run JavaScriptCore tests
https://bugs.webkit.org/show_bug.cgi?id=162458
Patch by Srinivasan Vijayaraghavan <svijayaraghavan@apple.com> on 2017-02-17
Reviewed by Alexey Proskuryakov.
* QueueStatusServer/config/queues.py: Add jsc-ews queue.
* QueueStatusServer/model/queuestatus.py:
(QueueStatus.did_skip): Returns whether patch was skipped, based on status.
* QueueStatusServer/handlers/statusbubble.py:
(StatusBubble._should_show_bubble_for): Add logic to hide jsc-ews bubble if the patch does not touch jsc.
* Scripts/webkitpy/common/checkout/scm/scm_mock.py:
(MockSCM.__init__): Add _mockChangedFiles variable.
(MockSCM.changed_files): Change to use _mockChangedFiles variables.
* Scripts/webkitpy/common/config/ews.json: Add config info for JSC EWS.
* Scripts/webkitpy/common/config/ports.py: Add support for JSC EWS in Mac Port.
(DeprecatedPort._append_build_style_flag): Helper function to append build_style to a command.
(DeprecatedPort.build_jsc_command): Added - command to build JSC only (quicker than building all of WebKit).
(DeprecatedPort.run_javascriptcore_tests_command): Allow JSC EWS to only run JSC tests.
(MacPort.run_webkit_tests_command): Check for JSC.
* Scripts/webkitpy/common/config/ports_mock.py:
(MockPort.run_javascriptcore_tests_command): Add build_style argument.
* Scripts/webkitpy/common/config/ports_unittest.py:
(DeprecatedPortTest.test_mac_port): Add unit tests for build-jsc shell command.
* Scripts/webkitpy/common/net/abstracttestresults.py: Added.
(AbstractTestResults): Abstract superclass of JSCTestResults and JSCTestResults.
(AbstractTestResults.test_results): Stub.
(AbstractTestResults.failing_tests): Stub.
(AbstractTestResults.did_exceed_test_failure_limit): Stub.
* Scripts/webkitpy/common/net/jsctestresults.py: Added.
(JSCTestResults): Added.
(JSCTestResults.intersection): Return a JSCTestResults object with failures common to both input objects.
(JSCTestResults.results_from_string): Use json library to parse results.
(JSCTestResults.__init__): Initialize members.
(JSCTestResults.equals): This enables unit testing.
(JSCTestResults.is_subset): Checks if one set of failures is a subset of another.
(JSCTestResults.all_passed): Getter.
(JSCTestResults.failing_tests): Getter.
(JSCTestResults.did_exceed_test_failure_limit): Getter.
* Scripts/webkitpy/common/net/jsctestresults_unittest.py: Added.
(JSCTestResultsTest): Class to test JSCTestResults.
(JSCTestResultsTest.test_results_from_string): Tests parsing.
(JSCTestResultsTest.test_intersection_api_tests): Unit test for intersection() class method.
(JSCTestResultsTest.test_intersection_stress_tests): Unit test for intersection() class method.
(JSCTestResultsTest.test_intersection_general_case): Unit test for intersection() class method.
* Scripts/webkitpy/port/base.py:
(Port.jsc_results_directory): Returns the directory for the JSC test results JSON file.
* Scripts/webkitpy/tool/bot/earlywarningsystemtask.py:
(EarlyWarningSystemTask.run): Add check_patch_relevance step.
* Scripts/webkitpy/tool/bot/jscews_unittest.py: Added.
(MockPatchAnalysisTask): Mocked-out version of PatchAnalysisTask that doesn't run commands.
(MockPatchAnalysisTask.__init__): Sets attributes.
(MockPatchAnalysisTask._test): Override to not run command.
(MockPatchAnalysisTask._build_and_test_without_patch): Override to not run command.
(MockPatchAnalysisTask.validate): Assume mocked patch is valid for purposes of testing retry logic.
(MockPatchAnalysisTask.test_run_count): Specific to the mocked version, to test retry logic.
(MockJSCEarlyWarningSystem): Mocked-out version of AbstarctEarlyWarningSystem so we can provide test results.
(MockJSCEarlyWarningSystem.__init__): Sets attributes, also sets group to jsc.
(MockJSCEarlyWarningSystem.test_results): Returns test results provided by us, instead of using a JSON reader.
(JSCEarlyWarningSystemTest): Class to test retry logic in below situations.
(JSCEarlyWarningSystemTest._create_task): Helper function to abstract out common code.
(JSCEarlyWarningSystemTest._results_indicate_all_passed): False if input is None or has failures, else True.
(JSCEarlyWarningSystemTest.test_success_case): Clean patch on clean tree.
(JSCEarlyWarningSystemTest.test_test_failure): Failed patch on clean tree.
(JSCEarlyWarningSystemTest.test_fix): Patch that fixes all tree redness.
(JSCEarlyWarningSystemTest.test_ineffective_patch): Patch that has same failures as tree.
(JSCEarlyWarningSystemTest.test_partially_effective_patch): Patch fixes some failures but adds no new failures.
(JSCEarlyWarningSystemTest.test_different_test_failures_in_patch_and_tree): Patch has some failures not in tree.
(JSCEarlyWarningSystemTest.test_first_results_could_not_be_read): Patch results not readable.
(JSCEarlyWarningSystemTest.test_second_results_could_not_be_read): Patch results not readable on second run.
(JSCEarlyWarningSystemTest.test_clean_results_could_not_be_read): Results from clean tree not readable.
(JSCEarlyWarningSystemTest.test_flaky_results_on_clean_tree_pass): Patch has one flake and no failures.
(JSCEarlyWarningSystemTest.test_flaky_results_on_clean_tree_pass_v2): Patch has one flake and no failures.
(JSCEarlyWarningSystemTest.test_flaky_results_on_clean_tree_failure): Patch has flakes and failed tests.
(JSCEarlyWarningSystemTest.test_flaky_results_on_red_tree_pass): Patch has same failures as tree, plus a flake.
* Scripts/webkitpy/tool/bot/jsctestresultsreader.py: Added.
(JSCTestResultsReader): Reads results file.
(JSCTestResultsReader.__init__): Sets attributes.
(JSCTestResultsReader._read_file_contents): Reads file.
(JSCTestResultsReader.results): Reads the results file and returns a JSCTestResults object.
* Scripts/webkitpy/tool/bot/patchanalysistask.py:
(PatchIsNotApplicable): Exception for when patch doesn't have relevant changes.
(PatchAnalysisTask._check_patch_relevance): Added.
(PatchAnalysisTask._build): Check for JSC.
(PatchAnalysisTask._build_without_patch): Check for JSC.
(PatchAnalysisTask._test): Check for JSC.
(PatchAnalysisTask._build_and_test_without_patch): Check for JSC.
(PatchAnalysisTask._retry_jsc_tests): Retry logic for JSC EWS.
(PatchAnalysisTask._retry_layout_tests): Retry logic for layout tests EWS.
(PatchAnalysisTask._test_patch): Add retry logic for JSC.
* Scripts/webkitpy/tool/commands/download.py:
(CheckPatchRelevance): Add check-patch-relevance command.
* Scripts/webkitpy/tool/commands/earlywarningsystem.py:
(AbstractEarlyWarningSystem._create_task): Abstract out to enable mocking.
(AbstractEarlyWarningSystem.begin_work_queue): Use JSCTestResultsReader not LayoutTestResultsReader in JSC EWS.
(AbstractEarlyWarningSystem.review_patch): Handle PatchIsApplicable.
(AbstractEarlyWarningSystem.test_results): _layout_test_results_reader -> _test_results_reader.
(AbstractEarlyWarningSystem.archive_last_test_results): _layout_test_results_reader -> _test_results_reader.
(AbstractEarlyWarningSystem.group): This attribute determines the type of EWS (eg. JSC).
(AbstractEarlyWarningSystem.load_ews_classes): Add _group, and make classes of type cls to enable mocking.
* Scripts/webkitpy/tool/commands/earlywarningsystem_unittest.py:
(TestEWS): Sample layout test EWS class used for unit testing.
(TestJSCEWS): Sample JSC EWS class used for unit testing.
(AbstractEarlyWarningSystemTest.test_failing_tests_message.TestEWS): Add _group.
(AbstractEarlyWarningSystemTest.test_failing_jsc_tests_message.TestEWS): Added test for jsc failures message.
(AbstractEarlyWarningSystemTest): Add _group variable.
(EarlyWarningSystemTest._default_expected_logs): Add check-patch-relevance step, inconclusive logs, and group.
(EarlyWarningSystemTest._test_ews): Add logs_are_conclusive option to pass through to default_expected_logs().
(EarlyWarningSystemTest.test_inconclusive_test_results): Test not removing patch from queue if not conclusive.
(MockAbstractEarlyWarningSystemForInconclusiveJSCResults): Added.
(MockAbstractEarlyWarningSystemForInconclusiveJSCResults._test_patch): Simulates running tests but not getting a conclusive result.
(MockEarlyWarningSystemTaskForInconclusiveJSCResults): Added.
(MockEarlyWarningSystemTaskForInconclusiveJSCResults._create_task): Use MockEarlyWarningSystemTask (not EarlyWarningSystemTask).
* Scripts/webkitpy/tool/steps/__init__.py: Add CheckPatchRelevance import.
* Scripts/webkitpy/tool/steps/build.py:
(Build.options): Check for JSC.
(Build.build): Check for JSC.
(Build.run): Check for JSC.
* Scripts/webkitpy/tool/steps/checkpatchrelevance.py: Added.
(CheckPatchRelevance): Added.
(CheckPatchRelevance._does_contain_change_in_paths): Abstract function to perform regex matching.
(CheckPatchRelevance.run): Check if changed files in patch belong in certain folders.
* Scripts/webkitpy/tool/steps/options.py:
(Options): Add --group command line option.
* Scripts/webkitpy/tool/steps/runtests.py:
(RunTests.options): Add group.
(RunTests.run): Check for JSC.
(RunTests._run_webkit_tests): Check for JSC.
(RunTests): Add _group attribute.
(RunTests._run_javascriptcore_tests): New.
* Scripts/webkitpy/tool/steps/steps_unittest.py: Unit tests.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkToolsChangeLog">trunk/Tools/ChangeLog</a></li>
<li><a href="#trunkToolsQueueStatusServerconfigmessagespy">trunk/Tools/QueueStatusServer/config/messages.py</a></li>
<li><a href="#trunkToolsQueueStatusServerconfigqueuespy">trunk/Tools/QueueStatusServer/config/queues.py</a></li>
<li><a href="#trunkToolsQueueStatusServerhandlersstatusbubblepy">trunk/Tools/QueueStatusServer/handlers/statusbubble.py</a></li>
<li><a href="#trunkToolsQueueStatusServermodelqueuespy">trunk/Tools/QueueStatusServer/model/queues.py</a></li>
<li><a href="#trunkToolsQueueStatusServermodelqueuestatuspy">trunk/Tools/QueueStatusServer/model/queuestatus.py</a></li>
<li><a href="#trunkToolsScriptswebkitpycommoncheckoutscmscm_mockpy">trunk/Tools/Scripts/webkitpy/common/checkout/scm/scm_mock.py</a></li>
<li><a href="#trunkToolsScriptswebkitpycommonconfigewsjson">trunk/Tools/Scripts/webkitpy/common/config/ews.json</a></li>
<li><a href="#trunkToolsScriptswebkitpycommonconfigportspy">trunk/Tools/Scripts/webkitpy/common/config/ports.py</a></li>
<li><a href="#trunkToolsScriptswebkitpycommonconfigports_mockpy">trunk/Tools/Scripts/webkitpy/common/config/ports_mock.py</a></li>
<li><a href="#trunkToolsScriptswebkitpycommonconfigports_unittestpy">trunk/Tools/Scripts/webkitpy/common/config/ports_unittest.py</a></li>
<li><a href="#trunkToolsScriptswebkitpycommonnetlayouttestresultspy">trunk/Tools/Scripts/webkitpy/common/net/layouttestresults.py</a></li>
<li><a href="#trunkToolsScriptswebkitpyportbasepy">trunk/Tools/Scripts/webkitpy/port/base.py</a></li>
<li><a href="#trunkToolsScriptswebkitpytoolbotearlywarningsystemtaskpy">trunk/Tools/Scripts/webkitpy/tool/bot/earlywarningsystemtask.py</a></li>
<li><a href="#trunkToolsScriptswebkitpytoolbotpatchanalysistaskpy">trunk/Tools/Scripts/webkitpy/tool/bot/patchanalysistask.py</a></li>
<li><a href="#trunkToolsScriptswebkitpytoolcommandsdownloadpy">trunk/Tools/Scripts/webkitpy/tool/commands/download.py</a></li>
<li><a href="#trunkToolsScriptswebkitpytoolcommandsdownload_unittestpy">trunk/Tools/Scripts/webkitpy/tool/commands/download_unittest.py</a></li>
<li><a href="#trunkToolsScriptswebkitpytoolcommandsearlywarningsystempy">trunk/Tools/Scripts/webkitpy/tool/commands/earlywarningsystem.py</a></li>
<li><a href="#trunkToolsScriptswebkitpytoolcommandsearlywarningsystem_unittestpy">trunk/Tools/Scripts/webkitpy/tool/commands/earlywarningsystem_unittest.py</a></li>
<li><a href="#trunkToolsScriptswebkitpytoolcommandsqueuespy">trunk/Tools/Scripts/webkitpy/tool/commands/queues.py</a></li>
<li><a href="#trunkToolsScriptswebkitpytoolsteps__init__py">trunk/Tools/Scripts/webkitpy/tool/steps/__init__.py</a></li>
<li><a href="#trunkToolsScriptswebkitpytoolstepsbuildpy">trunk/Tools/Scripts/webkitpy/tool/steps/build.py</a></li>
<li><a href="#trunkToolsScriptswebkitpytoolstepsoptionspy">trunk/Tools/Scripts/webkitpy/tool/steps/options.py</a></li>
<li><a href="#trunkToolsScriptswebkitpytoolstepsruntestspy">trunk/Tools/Scripts/webkitpy/tool/steps/runtests.py</a></li>
<li><a href="#trunkToolsScriptswebkitpytoolstepsruntests_unittestpy">trunk/Tools/Scripts/webkitpy/tool/steps/runtests_unittest.py</a></li>
<li><a href="#trunkToolsScriptswebkitpytoolstepssteps_unittestpy">trunk/Tools/Scripts/webkitpy/tool/steps/steps_unittest.py</a></li>
</ul>
<h3>Added Paths</h3>
<ul>
<li><a href="#trunkToolsScriptswebkitpycommonnetabstracttestresultspy">trunk/Tools/Scripts/webkitpy/common/net/abstracttestresults.py</a></li>
<li><a href="#trunkToolsScriptswebkitpycommonnetjsctestresultspy">trunk/Tools/Scripts/webkitpy/common/net/jsctestresults.py</a></li>
<li><a href="#trunkToolsScriptswebkitpycommonnetjsctestresults_unittestpy">trunk/Tools/Scripts/webkitpy/common/net/jsctestresults_unittest.py</a></li>
<li><a href="#trunkToolsScriptswebkitpytoolbotjscews_unittestpy">trunk/Tools/Scripts/webkitpy/tool/bot/jscews_unittest.py</a></li>
<li><a href="#trunkToolsScriptswebkitpytoolbotjsctestresultsreaderpy">trunk/Tools/Scripts/webkitpy/tool/bot/jsctestresultsreader.py</a></li>
<li><a href="#trunkToolsScriptswebkitpytoolstepscheckpatchrelevancepy">trunk/Tools/Scripts/webkitpy/tool/steps/checkpatchrelevance.py</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkToolsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Tools/ChangeLog (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/ChangeLog 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/ChangeLog 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,3 +1,136 @@
</span><ins>+2017-02-17 Srinivasan Vijayaraghavan <svijayaraghavan@apple.com>
+
+ EWS should run JavaScriptCore tests
+ https://bugs.webkit.org/show_bug.cgi?id=162458
+
+ Reviewed by Alexey Proskuryakov.
+
+ * QueueStatusServer/config/queues.py: Add jsc-ews queue.
+ * QueueStatusServer/model/queuestatus.py:
+ (QueueStatus.did_skip): Returns whether patch was skipped, based on status.
+ * QueueStatusServer/handlers/statusbubble.py:
+ (StatusBubble._should_show_bubble_for): Add logic to hide jsc-ews bubble if the patch does not touch jsc.
+ * Scripts/webkitpy/common/checkout/scm/scm_mock.py:
+ (MockSCM.__init__): Add _mockChangedFiles variable.
+ (MockSCM.changed_files): Change to use _mockChangedFiles variables.
+ * Scripts/webkitpy/common/config/ews.json: Add config info for JSC EWS.
+ * Scripts/webkitpy/common/config/ports.py: Add support for JSC EWS in Mac Port.
+ (DeprecatedPort._append_build_style_flag): Helper function to append build_style to a command.
+ (DeprecatedPort.build_jsc_command): Added - command to build JSC only (quicker than building all of WebKit).
+ (DeprecatedPort.run_javascriptcore_tests_command): Allow JSC EWS to only run JSC tests.
+ (MacPort.run_webkit_tests_command): Check for JSC.
+ * Scripts/webkitpy/common/config/ports_mock.py:
+ (MockPort.run_javascriptcore_tests_command): Add build_style argument.
+ * Scripts/webkitpy/common/config/ports_unittest.py:
+ (DeprecatedPortTest.test_mac_port): Add unit tests for build-jsc shell command.
+ * Scripts/webkitpy/common/net/abstracttestresults.py: Added.
+ (AbstractTestResults): Abstract superclass of JSCTestResults and JSCTestResults.
+ (AbstractTestResults.test_results): Stub.
+ (AbstractTestResults.failing_tests): Stub.
+ (AbstractTestResults.did_exceed_test_failure_limit): Stub.
+ * Scripts/webkitpy/common/net/jsctestresults.py: Added.
+ (JSCTestResults): Added.
+ (JSCTestResults.intersection): Return a JSCTestResults object with failures common to both input objects.
+ (JSCTestResults.results_from_string): Use json library to parse results.
+ (JSCTestResults.__init__): Initialize members.
+ (JSCTestResults.equals): This enables unit testing.
+ (JSCTestResults.is_subset): Checks if one set of failures is a subset of another.
+ (JSCTestResults.all_passed): Getter.
+ (JSCTestResults.failing_tests): Getter.
+ (JSCTestResults.did_exceed_test_failure_limit): Getter.
+ * Scripts/webkitpy/common/net/jsctestresults_unittest.py: Added.
+ (JSCTestResultsTest): Class to test JSCTestResults.
+ (JSCTestResultsTest.test_results_from_string): Tests parsing.
+ (JSCTestResultsTest.test_intersection_api_tests): Unit test for intersection() class method.
+ (JSCTestResultsTest.test_intersection_stress_tests): Unit test for intersection() class method.
+ (JSCTestResultsTest.test_intersection_general_case): Unit test for intersection() class method.
+ * Scripts/webkitpy/port/base.py:
+ (Port.jsc_results_directory): Returns the directory for the JSC test results JSON file.
+ * Scripts/webkitpy/tool/bot/earlywarningsystemtask.py:
+ (EarlyWarningSystemTask.run): Add check_patch_relevance step.
+ * Scripts/webkitpy/tool/bot/jscews_unittest.py: Added.
+ (MockPatchAnalysisTask): Mocked-out version of PatchAnalysisTask that doesn't run commands.
+ (MockPatchAnalysisTask.__init__): Sets attributes.
+ (MockPatchAnalysisTask._test): Override to not run command.
+ (MockPatchAnalysisTask._build_and_test_without_patch): Override to not run command.
+ (MockPatchAnalysisTask.validate): Assume mocked patch is valid for purposes of testing retry logic.
+ (MockPatchAnalysisTask.test_run_count): Specific to the mocked version, to test retry logic.
+ (MockJSCEarlyWarningSystem): Mocked-out version of AbstarctEarlyWarningSystem so we can provide test results.
+ (MockJSCEarlyWarningSystem.__init__): Sets attributes, also sets group to jsc.
+ (MockJSCEarlyWarningSystem.test_results): Returns test results provided by us, instead of using a JSON reader.
+ (JSCEarlyWarningSystemTest): Class to test retry logic in below situations.
+ (JSCEarlyWarningSystemTest._create_task): Helper function to abstract out common code.
+ (JSCEarlyWarningSystemTest._results_indicate_all_passed): False if input is None or has failures, else True.
+ (JSCEarlyWarningSystemTest.test_success_case): Clean patch on clean tree.
+ (JSCEarlyWarningSystemTest.test_test_failure): Failed patch on clean tree.
+ (JSCEarlyWarningSystemTest.test_fix): Patch that fixes all tree redness.
+ (JSCEarlyWarningSystemTest.test_ineffective_patch): Patch that has same failures as tree.
+ (JSCEarlyWarningSystemTest.test_partially_effective_patch): Patch fixes some failures but adds no new failures.
+ (JSCEarlyWarningSystemTest.test_different_test_failures_in_patch_and_tree): Patch has some failures not in tree.
+ (JSCEarlyWarningSystemTest.test_first_results_could_not_be_read): Patch results not readable.
+ (JSCEarlyWarningSystemTest.test_second_results_could_not_be_read): Patch results not readable on second run.
+ (JSCEarlyWarningSystemTest.test_clean_results_could_not_be_read): Results from clean tree not readable.
+ (JSCEarlyWarningSystemTest.test_flaky_results_on_clean_tree_pass): Patch has one flake and no failures.
+ (JSCEarlyWarningSystemTest.test_flaky_results_on_clean_tree_pass_v2): Patch has one flake and no failures.
+ (JSCEarlyWarningSystemTest.test_flaky_results_on_clean_tree_failure): Patch has flakes and failed tests.
+ (JSCEarlyWarningSystemTest.test_flaky_results_on_red_tree_pass): Patch has same failures as tree, plus a flake.
+ * Scripts/webkitpy/tool/bot/jsctestresultsreader.py: Added.
+ (JSCTestResultsReader): Reads results file.
+ (JSCTestResultsReader.__init__): Sets attributes.
+ (JSCTestResultsReader._read_file_contents): Reads file.
+ (JSCTestResultsReader.results): Reads the results file and returns a JSCTestResults object.
+ * Scripts/webkitpy/tool/bot/patchanalysistask.py:
+ (PatchIsNotApplicable): Exception for when patch doesn't have relevant changes.
+ (PatchAnalysisTask._check_patch_relevance): Added.
+ (PatchAnalysisTask._build): Check for JSC.
+ (PatchAnalysisTask._build_without_patch): Check for JSC.
+ (PatchAnalysisTask._test): Check for JSC.
+ (PatchAnalysisTask._build_and_test_without_patch): Check for JSC.
+ (PatchAnalysisTask._retry_jsc_tests): Retry logic for JSC EWS.
+ (PatchAnalysisTask._retry_layout_tests): Retry logic for layout tests EWS.
+ (PatchAnalysisTask._test_patch): Add retry logic for JSC.
+ * Scripts/webkitpy/tool/commands/download.py:
+ (CheckPatchRelevance): Add check-patch-relevance command.
+ * Scripts/webkitpy/tool/commands/earlywarningsystem.py:
+ (AbstractEarlyWarningSystem._create_task): Abstract out to enable mocking.
+ (AbstractEarlyWarningSystem.begin_work_queue): Use JSCTestResultsReader not LayoutTestResultsReader in JSC EWS.
+ (AbstractEarlyWarningSystem.review_patch): Handle PatchIsApplicable.
+ (AbstractEarlyWarningSystem.test_results): _layout_test_results_reader -> _test_results_reader.
+ (AbstractEarlyWarningSystem.archive_last_test_results): _layout_test_results_reader -> _test_results_reader.
+ (AbstractEarlyWarningSystem.group): This attribute determines the type of EWS (eg. JSC).
+ (AbstractEarlyWarningSystem.load_ews_classes): Add _group, and make classes of type cls to enable mocking.
+ * Scripts/webkitpy/tool/commands/earlywarningsystem_unittest.py:
+ (TestEWS): Sample layout test EWS class used for unit testing.
+ (TestJSCEWS): Sample JSC EWS class used for unit testing.
+ (AbstractEarlyWarningSystemTest.test_failing_tests_message.TestEWS): Add _group.
+ (AbstractEarlyWarningSystemTest.test_failing_jsc_tests_message.TestEWS): Added test for jsc failures message.
+ (AbstractEarlyWarningSystemTest): Add _group variable.
+ (EarlyWarningSystemTest._default_expected_logs): Add check-patch-relevance step, inconclusive logs, and group.
+ (EarlyWarningSystemTest._test_ews): Add logs_are_conclusive option to pass through to default_expected_logs().
+ (EarlyWarningSystemTest.test_inconclusive_test_results): Test not removing patch from queue if not conclusive.
+ (MockAbstractEarlyWarningSystemForInconclusiveJSCResults): Added.
+ (MockAbstractEarlyWarningSystemForInconclusiveJSCResults._test_patch): Simulates running tests but not getting a conclusive result.
+ (MockEarlyWarningSystemTaskForInconclusiveJSCResults): Added.
+ (MockEarlyWarningSystemTaskForInconclusiveJSCResults._create_task): Use MockEarlyWarningSystemTask (not EarlyWarningSystemTask).
+ * Scripts/webkitpy/tool/steps/__init__.py: Add CheckPatchRelevance import.
+ * Scripts/webkitpy/tool/steps/build.py:
+ (Build.options): Check for JSC.
+ (Build.build): Check for JSC.
+ (Build.run): Check for JSC.
+ * Scripts/webkitpy/tool/steps/checkpatchrelevance.py: Added.
+ (CheckPatchRelevance): Added.
+ (CheckPatchRelevance._does_contain_change_in_paths): Abstract function to perform regex matching.
+ (CheckPatchRelevance.run): Check if changed files in patch belong in certain folders.
+ * Scripts/webkitpy/tool/steps/options.py:
+ (Options): Add --group command line option.
+ * Scripts/webkitpy/tool/steps/runtests.py:
+ (RunTests.options): Add group.
+ (RunTests.run): Check for JSC.
+ (RunTests._run_webkit_tests): Check for JSC.
+ (RunTests): Add _group attribute.
+ (RunTests._run_javascriptcore_tests): New.
+ * Scripts/webkitpy/tool/steps/steps_unittest.py: Unit tests.
+
</ins><span class="cx"> 2017-02-17 Aakash Jain <aakash_jain@apple.com>
</span><span class="cx">
</span><span class="cx"> Fix tools that were broken by Efl removal
</span></span></pre></div>
<a id="trunkToolsQueueStatusServerconfigmessagespy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/QueueStatusServer/config/messages.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/QueueStatusServer/config/messages.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/QueueStatusServer/config/messages.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,4 +1,5 @@
</span><span class="cx"> # Copyright (C) 2013 Google Inc. All rights reserved.
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -27,6 +28,7 @@
</span><span class="cx"> # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
</span><span class="cx">
</span><span class="cx"> # These must be in sync with webkit-patch's AbstractQueue.
</span><ins>+skip_status = "Skip"
</ins><span class="cx"> pass_status = "Pass"
</span><span class="cx"> fail_status = "Fail"
</span><span class="cx"> error_status = "Error"
</span></span></pre></div>
<a id="trunkToolsQueueStatusServerconfigqueuespy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/QueueStatusServer/config/queues.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/QueueStatusServer/config/queues.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/QueueStatusServer/config/queues.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,4 +1,4 @@
</span><del>-# Copyright (C) 2014 Apple Inc. All rights reserved.
</del><ins>+# Copyright (C) 2014, 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> # Copyright (C) 2013 Google Inc. All rights reserved.
</span><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="lines">@@ -36,6 +36,7 @@
</span><span class="cx"> "gtk-wk2-ews",
</span><span class="cx"> "ios-ews",
</span><span class="cx"> "ios-sim-ews",
</span><ins>+ "jsc-ews",
</ins><span class="cx"> "mac-ews",
</span><span class="cx"> "mac-wk2-ews",
</span><span class="cx"> "mac-debug-ews",
</span></span></pre></div>
<a id="trunkToolsQueueStatusServerhandlersstatusbubblepy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/QueueStatusServer/handlers/statusbubble.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/QueueStatusServer/handlers/statusbubble.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/QueueStatusServer/handlers/statusbubble.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,4 +1,5 @@
</span><span class="cx"> # Copyright (C) 2009 Google Inc. All rights reserved.
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -182,12 +183,16 @@
</span><span class="cx"> return bubble
</span><span class="cx">
</span><span class="cx"> def _should_show_bubble_for(self, attachment, queue):
</span><del>- # Any pending queue is shown.
</del><ins>+ # Any pending queue is shown.
</ins><span class="cx"> if attachment.position_in_queue(queue):
</span><span class="cx"> return True
</span><del>- # EWS queues are also shown when complete.
- return bool(queue.is_ews() and attachment.status_for_queue(queue))
</del><span class="cx">
</span><ins>+ if not queue.is_ews():
+ return False
+
+ status = attachment.status_for_queue(queue)
+ return bool(status and not status.did_skip())
+
</ins><span class="cx"> def _build_bubbles_for_attachment(self, attachment):
</span><span class="cx"> show_submit_to_ews = True
</span><span class="cx"> bubbles = []
</span></span></pre></div>
<a id="trunkToolsQueueStatusServermodelqueuespy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/QueueStatusServer/model/queues.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/QueueStatusServer/model/queues.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/QueueStatusServer/model/queues.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -78,6 +78,7 @@
</span><span class="cx"> display_name = display_name.replace("Wk2", "WK2")
</span><span class="cx"> display_name = display_name.replace("Ews", "EWS")
</span><span class="cx"> display_name = display_name.replace("Ios", "iOS")
</span><ins>+ display_name = display_name.replace("Jsc", "JSC")
</ins><span class="cx"> return display_name
</span><span class="cx">
</span><span class="cx"> _dash_regexp = re.compile("-")
</span></span></pre></div>
<a id="trunkToolsQueueStatusServermodelqueuestatuspy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/QueueStatusServer/model/queuestatus.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/QueueStatusServer/model/queuestatus.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/QueueStatusServer/model/queuestatus.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,4 +1,5 @@
</span><span class="cx"> # Copyright (C) 2013 Google Inc. All rights reserved.
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -40,3 +41,6 @@
</span><span class="cx"> message = db.StringProperty(multiline=True)
</span><span class="cx"> date = db.DateTimeProperty(auto_now_add=True)
</span><span class="cx"> results_file = db.BlobProperty()
</span><ins>+
+ def did_skip(self):
+ return self.message == messages.skip_status
</ins></span></pre></div>
<a id="trunkToolsScriptswebkitpycommoncheckoutscmscm_mockpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/common/checkout/scm/scm_mock.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/common/checkout/scm/scm_mock.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/common/checkout/scm/scm_mock.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,4 +1,5 @@
</span><span class="cx"> # Copyright (C) 2011 Google Inc. All rights reserved.
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -37,6 +38,7 @@
</span><span class="cx"> self.added_paths = set()
</span><span class="cx"> self._filesystem = filesystem or MockFileSystem()
</span><span class="cx"> self._executive = executive or MockExecutive()
</span><ins>+ self._mockChangedFiles = ["MockFile1"]
</ins><span class="cx">
</span><span class="cx"> def add(self, destination_path):
</span><span class="cx"> self.add_list([destination_path])
</span><span class="lines">@@ -71,7 +73,7 @@
</span><span class="cx"> return self._filesystem.join(self.checkout_root, *comps)
</span><span class="cx">
</span><span class="cx"> def changed_files(self, git_commit=None):
</span><del>- return ["MockFile1"]
</del><ins>+ return self._mockChangedFiles
</ins><span class="cx">
</span><span class="cx"> def changed_files_for_revision(self, revision):
</span><span class="cx"> return ["MockFile1"]
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpycommonconfigewsjson"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/common/config/ews.json (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/common/config/ews.json 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/common/config/ews.json 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -44,5 +44,11 @@
</span><span class="cx"> "port": "mac",
</span><span class="cx"> "name": "mac-32bit-ews",
</span><span class="cx"> "architecture": "i386"
</span><ins>+ },
+ "JSC EWS": {
+ "port": "mac",
+ "name": "jsc-ews",
+ "group": "jsc",
+ "runTests": true
</ins><span class="cx"> }
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpycommonconfigportspy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/common/config/ports.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/common/config/ports.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/common/config/ports.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,6 +1,6 @@
</span><span class="cx"> # Copyright (C) 2009, Google Inc. All rights reserved.
</span><span class="cx"> # Copyright (C) 2013 Nokia Corporation and/or its subsidiary(-ies).
</span><del>-# Copyright (C) 2015 Apple Inc. All rights reserved.
</del><ins>+# Copyright (C) 2015, 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -95,24 +95,29 @@
</span><span class="cx"> def prepare_changelog_command(self):
</span><span class="cx"> return self.script_shell_command("prepare-ChangeLog")
</span><span class="cx">
</span><del>- def build_webkit_command(self, build_style=None):
- command = self.script_shell_command("build-webkit")
</del><ins>+ def _append_build_style_flag(self, command, build_style):
</ins><span class="cx"> if build_style == "debug":
</span><span class="cx"> command.append("--debug")
</span><del>- if build_style == "release":
</del><ins>+ elif build_style == "release":
</ins><span class="cx"> command.append("--release")
</span><span class="cx"> return command
</span><span class="cx">
</span><del>- def run_javascriptcore_tests_command(self):
- return self.script_shell_command("run-javascriptcore-tests")
</del><ins>+ def build_webkit_command(self, build_style=None):
+ command = self.script_shell_command("build-webkit")
+ return self._append_build_style_flag(command, build_style)
</ins><span class="cx">
</span><ins>+ def build_jsc_command(self, build_style=None):
+ command = self.script_shell_command("build-jsc")
+ return self._append_build_style_flag(command, build_style)
+
+ def run_javascriptcore_tests_command(self, build_style=None):
+ command = self.script_shell_command("run-javascriptcore-tests")
+ command.append("--no-fail-fast")
+ return self._append_build_style_flag(command, build_style)
+
</ins><span class="cx"> def run_webkit_tests_command(self, build_style=None):
</span><span class="cx"> command = self.script_shell_command("run-webkit-tests")
</span><del>- if build_style == "debug":
- command.append("--debug")
- if build_style == "release":
- command.append("--release")
- return command
</del><ins>+ return self._append_build_style_flag(command, build_style)
</ins><span class="cx">
</span><span class="cx"> def run_python_unittests_command(self):
</span><span class="cx"> return self.script_shell_command("test-webkitpy")
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpycommonconfigports_mockpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/common/config/ports_mock.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/common/config/ports_mock.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/common/config/ports_mock.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,4 +1,5 @@
</span><span class="cx"> # Copyright (C) 2011 Google Inc. All rights reserved.
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -49,7 +50,7 @@
</span><span class="cx"> def run_perl_unittests_command(self):
</span><span class="cx"> return ['mock-test-webkitperl']
</span><span class="cx">
</span><del>- def run_javascriptcore_tests_command(self):
</del><ins>+ def run_javascriptcore_tests_command(self, build_style=None):
</ins><span class="cx"> return ['mock-run-javacriptcore-tests']
</span><span class="cx">
</span><span class="cx"> def run_webkit_tests_command(self, build_style=None):
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpycommonconfigports_unittestpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/common/config/ports_unittest.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/common/config/ports_unittest.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/common/config/ports_unittest.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,4 +1,5 @@
</span><span class="cx"> # Copyright (c) 2009, Google Inc. All rights reserved.
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -39,6 +40,10 @@
</span><span class="cx"> self.assertEqual(MacPort().build_webkit_command(build_style="debug"), DeprecatedPort().script_shell_command("build-webkit") + ["--debug"])
</span><span class="cx"> self.assertEqual(MacPort().build_webkit_command(build_style="release"), DeprecatedPort().script_shell_command("build-webkit") + ["--release"])
</span><span class="cx">
</span><ins>+ self.assertEqual(MacPort().build_jsc_command(), DeprecatedPort().script_shell_command("build-jsc"))
+ self.assertEqual(MacPort().build_jsc_command(build_style="release"), DeprecatedPort().script_shell_command("build-jsc") + ["--release"])
+ self.assertEqual(MacPort().build_jsc_command(build_style="debug"), DeprecatedPort().script_shell_command("build-jsc") + ["--debug"])
+
</ins><span class="cx"> def test_gtk_wk2_port(self):
</span><span class="cx"> self.assertEqual(GtkWK2Port().flag(), "--port=gtk-wk2")
</span><span class="cx"> self.assertEqual(GtkWK2Port().run_webkit_tests_command(), DeprecatedPort().script_shell_command("run-webkit-tests") + ["--gtk"])
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpycommonnetabstracttestresultspy"></a>
<div class="addfile"><h4>Added: trunk/Tools/Scripts/webkitpy/common/net/abstracttestresults.py (0 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/common/net/abstracttestresults.py (rev 0)
+++ trunk/Tools/Scripts/webkitpy/common/net/abstracttestresults.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -0,0 +1,33 @@
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS BE LIABLE FOR
+# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import logging
+
+_log = logging.getLogger(__name__)
+
+
+class AbstractTestResults(object):
+ def failing_tests(self):
+ raise NotImplementedError("subclasses must implement")
+
+ def did_exceed_test_failure_limit(self):
+ raise NotImplementedError("subclasses must implement")
</ins></span></pre></div>
<a id="trunkToolsScriptswebkitpycommonnetjsctestresultspy"></a>
<div class="addfile"><h4>Added: trunk/Tools/Scripts/webkitpy/common/net/jsctestresults.py (0 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/common/net/jsctestresults.py (rev 0)
+++ trunk/Tools/Scripts/webkitpy/common/net/jsctestresults.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -0,0 +1,77 @@
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS BE LIABLE FOR
+# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import json
+import logging
+
+from webkitpy.common.net.abstracttestresults import AbstractTestResults
+
+_log = logging.getLogger(__name__)
+
+
+class JSCTestResults(AbstractTestResults):
+ def __init__(self, all_api_tests_passed, stress_test_failures):
+ self._all_api_tests_passed = all_api_tests_passed
+ self._stress_test_failures = stress_test_failures
+
+ self._failing_test_names = stress_test_failures[:]
+ if not self._all_api_tests_passed:
+ self._failing_test_names.append('apiTests')
+
+ @classmethod
+ def intersection(cls, first, second):
+ intersection_api_tests_passed = first._all_api_tests_passed or second._all_api_tests_passed
+ intersection_stress_test_failures = list(set(first._stress_test_failures) & set(second._stress_test_failures))
+ return cls(intersection_api_tests_passed, intersection_stress_test_failures)
+
+ @classmethod
+ def results_from_string(cls, string):
+ if not string:
+ return None
+
+ try:
+ parsed_results = json.loads(string)
+ except ValueError:
+ _log.error('Invalid JSON results')
+ return None
+
+ if 'allApiTestsPassed' not in parsed_results or 'stressTestFailures' not in parsed_results:
+ return None
+
+ return cls(parsed_results['allApiTestsPassed'], parsed_results['stressTestFailures'])
+
+ def equals(self, other):
+ return (self._all_api_tests_passed == other._all_api_tests_passed and
+ set(self._stress_test_failures) == set(other._stress_test_failures))
+
+ def is_subset(self, other):
+ return set(self._failing_test_names) <= set(other._failing_test_names)
+
+ def all_passed(self):
+ return self._all_api_tests_passed and not self._stress_test_failures
+
+ def failing_tests(self):
+ return self._failing_test_names
+
+ # No defined failure limit for JSC.
+ def did_exceed_test_failure_limit(self):
+ return False
</ins></span></pre></div>
<a id="trunkToolsScriptswebkitpycommonnetjsctestresults_unittestpy"></a>
<div class="addfile"><h4>Added: trunk/Tools/Scripts/webkitpy/common/net/jsctestresults_unittest.py (0 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/common/net/jsctestresults_unittest.py (rev 0)
+++ trunk/Tools/Scripts/webkitpy/common/net/jsctestresults_unittest.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -0,0 +1,75 @@
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS BE LIABLE FOR
+# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import unittest
+
+from webkitpy.common.net.jsctestresults import JSCTestResults
+
+
+class JSCTestResultsTest(unittest.TestCase):
+ def test_results_from_string(self):
+ none_item = None
+ empty_json = ''
+ invalid_json = '{"allApiTestsPassed":'
+ incomplete_json_v1 = '{"allApiTestsPassed": true}'
+ incomplete_json_v2 = '{"stressTestFailures":[]}'
+ self.assertEqual(None, JSCTestResults.results_from_string(none_item))
+ self.assertEqual(None, JSCTestResults.results_from_string(empty_json))
+ self.assertEqual(None, JSCTestResults.results_from_string(invalid_json))
+ self.assertEqual(None, JSCTestResults.results_from_string(incomplete_json_v1))
+ self.assertEqual(None, JSCTestResults.results_from_string(incomplete_json_v2))
+
+ no_failures_string = '{"allApiTestsPassed": true, "stressTestFailures":[]}'
+ no_failures_results = JSCTestResults(True, [])
+ self.assertTrue(no_failures_results.equals(JSCTestResults.results_from_string(no_failures_string)))
+
+ api_test_failures_string = '{"allApiTestsPassed": false, "stressTestFailures":[]}'
+ api_test_failures_results = JSCTestResults(False, [])
+ self.assertTrue(api_test_failures_results.equals(JSCTestResults.results_from_string(api_test_failures_string)))
+
+ many_failures_string = '{"allApiTestsPassed": false, "stressTestFailures":["es6.yaml/es6/typed_arrays_Int16Array.js.default", "es6.yaml/es6/typed_arrays_Int8Array.js.default"]}'
+ many_failures_results = JSCTestResults(False, ["es6.yaml/es6/typed_arrays_Int16Array.js.default", "es6.yaml/es6/typed_arrays_Int8Array.js.default"])
+ self.assertTrue(many_failures_results.equals(JSCTestResults.results_from_string(many_failures_string)))
+
+ self.assertFalse(no_failures_results == api_test_failures_results)
+ self.assertFalse(api_test_failures_results == many_failures_results)
+
+ def test_intersection_api_tests(self):
+ results1 = JSCTestResults(False, [])
+ results2 = JSCTestResults(True, [])
+
+ expected_intersection = JSCTestResults(True, [])
+ self.assertTrue(expected_intersection.equals(JSCTestResults.intersection(results1, results2)))
+
+ def test_intersection_stress_tests(self):
+ results1 = JSCTestResults(True, ['failure1', 'failure2'])
+ results2 = JSCTestResults(True, ['failure1', 'failure3'])
+
+ expected_intersection = JSCTestResults(True, ['failure1'])
+ self.assertTrue(expected_intersection.equals(JSCTestResults.intersection(results1, results2)))
+
+ def test_intersection_general_case(self):
+ results1 = JSCTestResults(True, ['failure1', 'failure2'])
+ results2 = JSCTestResults(False, ['failure1'])
+
+ expected_intersection = JSCTestResults(True, ['failure1'])
+ self.assertTrue(expected_intersection.equals(JSCTestResults.intersection(results1, results2)))
</ins></span></pre></div>
<a id="trunkToolsScriptswebkitpycommonnetlayouttestresultspy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/common/net/layouttestresults.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/common/net/layouttestresults.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/common/net/layouttestresults.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -28,6 +28,7 @@
</span><span class="cx">
</span><span class="cx"> import logging
</span><span class="cx">
</span><ins>+from webkitpy.common.net.abstracttestresults import AbstractTestResults
</ins><span class="cx"> from webkitpy.common.net.resultsjsonparser import ParsedJSONResults
</span><span class="cx"> from webkitpy.thirdparty.BeautifulSoup import BeautifulSoup, SoupStrainer
</span><span class="cx"> from webkitpy.layout_tests.models import test_results
</span><span class="lines">@@ -45,7 +46,7 @@
</span><span class="cx"> # FIXME: This should be unified with ResultsSummary or other NRWT layout tests code
</span><span class="cx"> # in the layout_tests package.
</span><span class="cx"> # This doesn't belong in common.net, but we don't have a better place for it yet.
</span><del>-class LayoutTestResults(object):
</del><ins>+class LayoutTestResults(AbstractTestResults):
</ins><span class="cx"> @classmethod
</span><span class="cx"> def results_from_string(cls, string):
</span><span class="cx"> if not string:
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpyportbasepy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/port/base.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/port/base.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/port/base.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -808,6 +808,9 @@
</span><span class="cx"> inverse of relative_test_filename()."""
</span><span class="cx"> return self._filesystem.join(self.layout_tests_dir(), test_name)
</span><span class="cx">
</span><ins>+ def jsc_results_directory(self):
+ return self._build_path()
+
</ins><span class="cx"> def results_directory(self):
</span><span class="cx"> """Absolute path to the place to store the test results (uses --results-directory)."""
</span><span class="cx"> if not self._results_directory:
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpytoolbotearlywarningsystemtaskpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/tool/bot/earlywarningsystemtask.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/tool/bot/earlywarningsystemtask.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/tool/bot/earlywarningsystemtask.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,4 +1,5 @@
</span><span class="cx"> # Copyright (c) 2011 Google Inc. All rights reserved.
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -26,7 +27,7 @@
</span><span class="cx"> # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
</span><span class="cx"> # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
</span><span class="cx">
</span><del>-from webkitpy.tool.bot.patchanalysistask import PatchAnalysisTask, PatchAnalysisTaskDelegate, UnableToApplyPatch, PatchIsNotValid
</del><ins>+from webkitpy.tool.bot.patchanalysistask import PatchAnalysisTask, PatchAnalysisTaskDelegate, UnableToApplyPatch, PatchIsNotValid, PatchIsNotApplicable
</ins><span class="cx">
</span><span class="cx">
</span><span class="cx"> class EarlyWarningSystemTaskDelegate(PatchAnalysisTaskDelegate):
</span><span class="lines">@@ -58,6 +59,8 @@
</span><span class="cx"> return False
</span><span class="cx"> if not self._apply():
</span><span class="cx"> raise UnableToApplyPatch(self._patch)
</span><ins>+ if not self._check_patch_relevance():
+ raise PatchIsNotApplicable(self._patch)
</ins><span class="cx"> if not self._build():
</span><span class="cx"> if not self._build_without_patch():
</span><span class="cx"> return False
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpytoolbotjscews_unittestpy"></a>
<div class="addfile"><h4>Added: trunk/Tools/Scripts/webkitpy/tool/bot/jscews_unittest.py (0 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/tool/bot/jscews_unittest.py (rev 0)
+++ trunk/Tools/Scripts/webkitpy/tool/bot/jscews_unittest.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -0,0 +1,218 @@
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS BE LIABLE FOR
+# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import logging
+import unittest
+
+from webkitpy.common.net.jsctestresults import JSCTestResults
+from webkitpy.common.system.executive import ScriptError
+from webkitpy.tool.bot.patchanalysistask import *
+from webkitpy.tool.commands.earlywarningsystem import AbstractEarlyWarningSystem
+from webkitpy.tool.mocktool import MockTool
+
+_log = logging.getLogger(__name__)
+
+
+class MockPatchAnalysisTask(PatchAnalysisTask):
+ def __init__(self, delegate, patch, patches_passed_all_tests):
+ self._delegate = delegate
+ self._patch = patch
+ self._script_error = None
+ self._results_archive_from_patch_test_run = None
+ self._results_from_patch_test_run = None
+ self.error = None
+ self._patches_passed_all_tests = patches_passed_all_tests
+ self._test_run_count = 0
+ self.failure_status_id = 0
+
+ def _test(self):
+ self._test_run_count += 1
+
+ if self._patches_passed_all_tests.pop() == True:
+ return True
+ self._script_error = ScriptError('Regression test')
+ return False
+
+ def _build_and_test_without_patch(self):
+ self._test_run_count += 1
+ return True
+
+ def validate(self):
+ return True
+
+ def test_run_count(self):
+ return self._test_run_count
+
+
+# This is the delegate to be used with MockPatchAnalysisTask, above.
+class MockJSCEarlyWarningSystem(AbstractEarlyWarningSystem):
+ def __init__(self, first_test_results, second_test_results, clean_test_results):
+ AbstractEarlyWarningSystem.__init__(self)
+ self._group = 'jsc'
+ self._results_in_order = [clean_test_results, second_test_results, first_test_results]
+
+ def test_results(self):
+ return self._results_in_order.pop()
+
+
+class JSCEarlyWarningSystemTest(unittest.TestCase):
+ def _results_indicate_all_passed(self, results):
+ if results == None:
+ return False
+ return results.all_passed()
+
+ def _create_task(self, first_test_results, second_test_results, clean_test_results):
+ queue = MockJSCEarlyWarningSystem(first_test_results, second_test_results, clean_test_results)
+ tool = MockTool(log_executive=True)
+ patch = tool.bugs.fetch_attachment(10000)
+ patches_passed_all_tests = map(self._results_indicate_all_passed, [second_test_results, first_test_results])
+ return MockPatchAnalysisTask(queue, patch, patches_passed_all_tests)
+
+ def test_success_case(self):
+ first_test_results = JSCTestResults(True, [])
+ second_test_results = JSCTestResults(True, [])
+ clean_test_results = JSCTestResults(True, [])
+ task = self._create_task(first_test_results, second_test_results, clean_test_results)
+
+ return_value = task._test_patch()
+ self.assertEqual(task.test_run_count(), 1)
+ self.assertTrue(return_value)
+
+ def test_test_failure(self):
+ first_test_results = JSCTestResults(True, ['Fail.js'])
+ second_test_results = JSCTestResults(True, ['Fail.js'])
+ clean_test_results = JSCTestResults(True, [])
+ task = self._create_task(first_test_results, second_test_results, clean_test_results)
+
+ with self.assertRaises(ScriptError):
+ return_value = task._test_patch()
+ self.assertEqual(task.test_run_count(), 3)
+
+ def test_fix(self):
+ first_test_results = JSCTestResults(True, [])
+ second_test_results = JSCTestResults(True, [])
+ clean_test_results = JSCTestResults(True, ['Fail.js'])
+ task = self._create_task(first_test_results, second_test_results, clean_test_results)
+
+ return_value = task._test_patch()
+ self.assertEqual(task.test_run_count(), 1)
+ self.assertTrue(return_value)
+
+ def test_ineffective_patch(self):
+ first_test_results = JSCTestResults(False, ['failure1.js', 'failure2.js'])
+ second_test_results = JSCTestResults(False, ['failure1.js', 'failure2.js'])
+ clean_test_results = JSCTestResults(False, ['failure1.js', 'failure2.js'])
+ task = self._create_task(first_test_results, second_test_results, clean_test_results)
+
+ return_value = task._test_patch()
+ self.assertEqual(task.test_run_count(), 3)
+ self.assertTrue(return_value)
+
+ def test_partially_effective_patch(self):
+ first_test_results = JSCTestResults(True, ['failure2.js'])
+ second_test_results = JSCTestResults(True, ['failure2.js'])
+ clean_test_results = JSCTestResults(False, ['failure1.js', 'failure2.js'])
+ task = self._create_task(first_test_results, second_test_results, clean_test_results)
+
+ return_value = task._test_patch()
+ self.assertEqual(task.test_run_count(), 3)
+ self.assertTrue(return_value)
+
+ def test_different_test_failures_in_patch_and_tree(self):
+ first_test_results = JSCTestResults(False, [])
+ second_test_results = JSCTestResults(False, [])
+ clean_test_results = JSCTestResults(True, ['failure1.js', 'failure2.js'])
+ task = self._create_task(first_test_results, second_test_results, clean_test_results)
+
+ with self.assertRaises(ScriptError):
+ return_value = task._test_patch()
+ self.assertEqual(task.test_run_count(), 3)
+
+ def test_first_results_could_not_be_read(self):
+ first_test_results = None
+ second_test_results = JSCTestResults(True, [])
+ clean_test_results = JSCTestResults(True, [])
+ task = self._create_task(first_test_results, second_test_results, clean_test_results)
+
+ return_value = task._test_patch()
+ self.assertEqual(task.test_run_count(), 1)
+ self.assertFalse(return_value)
+
+ def test_second_results_could_not_be_read(self):
+ first_test_results = JSCTestResults(False, ['failure1.js', 'failure2.js'])
+ second_test_results = None
+ clean_test_results = JSCTestResults(True, [])
+ task = self._create_task(first_test_results, second_test_results, clean_test_results)
+
+ return_value = task._test_patch()
+ self.assertEqual(task.test_run_count(), 2)
+ self.assertFalse(return_value)
+
+ def test_clean_results_could_not_be_read(self):
+ first_test_results = JSCTestResults(True, ['failure2.js'])
+ second_test_results = JSCTestResults(True, ['failure2.js'])
+ clean_test_results = None
+ task = self._create_task(first_test_results, second_test_results, clean_test_results)
+
+ return_value = task._test_patch()
+ self.assertEqual(task.test_run_count(), 3)
+ self.assertFalse(return_value)
+
+ def test_flaky_results_on_clean_tree_pass(self):
+ first_test_results = JSCTestResults(True, ['failure2.js'])
+ second_test_results = JSCTestResults(True, [])
+ clean_test_results = JSCTestResults(True, [])
+ task = self._create_task(first_test_results, second_test_results, clean_test_results)
+
+ return_value = task._test_patch()
+ self.assertTrue(return_value)
+ self.assertEqual(task.test_run_count(), 2)
+
+ def test_flaky_results_on_clean_tree_pass_v2(self):
+ first_test_results = JSCTestResults(True, [])
+ second_test_results = JSCTestResults(True, ['failure2.js'])
+ clean_test_results = JSCTestResults(True, [])
+ task = self._create_task(first_test_results, second_test_results, clean_test_results)
+
+ return_value = task._test_patch()
+ self.assertTrue(return_value)
+ self.assertEqual(task.test_run_count(), 1)
+
+ def test_flaky_results_on_clean_tree_failure(self):
+ first_test_results = JSCTestResults(False, ['failure1.js', 'failure2.js'])
+ second_test_results = JSCTestResults(True, ['failure2.js'])
+ clean_test_results = JSCTestResults(True, [])
+ task = self._create_task(first_test_results, second_test_results, clean_test_results)
+
+ with self.assertRaises(ScriptError):
+ task._test_patch()
+ self.assertEqual(task.test_run_count(), 3)
+
+ def test_flaky_results_on_red_tree_pass(self):
+ first_test_results = JSCTestResults(True, ['failure1.js'])
+ second_test_results = JSCTestResults(True, ['failure1.js', 'failure2.js'])
+ clean_test_results = JSCTestResults(True, ['failure1.js'])
+ task = self._create_task(first_test_results, second_test_results, clean_test_results)
+
+ return_value = task._test_patch()
+ self.assertTrue(return_value)
+ self.assertEqual(task.test_run_count(), 3)
</ins></span></pre></div>
<a id="trunkToolsScriptswebkitpytoolbotjsctestresultsreaderpy"></a>
<div class="addfile"><h4>Added: trunk/Tools/Scripts/webkitpy/tool/bot/jsctestresultsreader.py (0 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/tool/bot/jsctestresultsreader.py (rev 0)
+++ trunk/Tools/Scripts/webkitpy/tool/bot/jsctestresultsreader.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -0,0 +1,44 @@
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS BE LIABLE FOR
+# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import logging
+
+from webkitpy.common.net.jsctestresults import JSCTestResults
+
+_log = logging.getLogger(__name__)
+
+
+class JSCTestResultsReader(object):
+ def __init__(self, host, results_directory):
+ self._host = host
+ self._results_directory = results_directory
+
+ def _read_file_contents(self, path):
+ try:
+ return self._host.filesystem.read_text_file(path)
+ except (IOError, KeyError):
+ return None
+
+ def results(self):
+ results_path = self._host.filesystem.join(self._results_directory, 'jsc_test_results.json')
+ contents = self._read_file_contents(results_path)
+ return JSCTestResults.results_from_string(contents)
</ins></span></pre></div>
<a id="trunkToolsScriptswebkitpytoolbotpatchanalysistaskpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/tool/bot/patchanalysistask.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/tool/bot/patchanalysistask.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/tool/bot/patchanalysistask.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,4 +1,5 @@
</span><span class="cx"> # Copyright (c) 2010 Google Inc. All rights reserved.
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -28,6 +29,7 @@
</span><span class="cx">
</span><span class="cx"> from webkitpy.common.system.executive import ScriptError
</span><span class="cx"> from webkitpy.common.net.layouttestresults import LayoutTestResults
</span><ins>+from webkitpy.common.net.jsctestresults import JSCTestResults
</ins><span class="cx">
</span><span class="cx">
</span><span class="cx"> class UnableToApplyPatch(Exception):
</span><span class="lines">@@ -43,6 +45,11 @@
</span><span class="cx"> self.failure_message = failure_message
</span><span class="cx">
</span><span class="cx">
</span><ins>+class PatchIsNotApplicable(Exception):
+ def __init__(self, patch):
+ Exception.__init__(self)
+ self.patch = patch
+
</ins><span class="cx"> class PatchAnalysisTaskDelegate(object):
</span><span class="cx"> def parent_command(self):
</span><span class="cx"> raise NotImplementedError("subclasses must implement")
</span><span class="lines">@@ -122,28 +129,44 @@
</span><span class="cx"> "Applied patch",
</span><span class="cx"> "Patch does not apply")
</span><span class="cx">
</span><ins>+ def _check_patch_relevance(self):
+ args = [
+ "check-patch-relevance",
+ ]
+
+ if hasattr(self._delegate, 'group'):
+ args.append("--group=%s" % self._delegate.group())
+
+ return self._run_command(args, "Checked relevance of patch", "Patch was not relevant")
+
</ins><span class="cx"> def _build(self):
</span><del>- return self._run_command([
</del><ins>+ args = [
</ins><span class="cx"> "build",
</span><span class="cx"> "--no-clean",
</span><span class="cx"> "--no-update",
</span><span class="cx"> "--build-style=%s" % self._delegate.build_style(),
</span><del>- ],
- "Built patch",
- "Patch does not build")
</del><ins>+ ]
</ins><span class="cx">
</span><ins>+ if hasattr(self._delegate, 'group'):
+ args.append("--group=%s" % self._delegate.group())
+
+ return self._run_command(args, "Built patch", "Patch does not build")
+
</ins><span class="cx"> def _build_without_patch(self):
</span><del>- return self._run_command([
</del><ins>+ args = [
</ins><span class="cx"> "build",
</span><span class="cx"> "--force-clean",
</span><span class="cx"> "--no-update",
</span><span class="cx"> "--build-style=%s" % self._delegate.build_style(),
</span><del>- ],
- "Able to build without patch",
- "Unable to build without patch")
</del><ins>+ ]
</ins><span class="cx">
</span><ins>+ if hasattr(self._delegate, 'group'):
+ args.append("--group=%s" % self._delegate.group())
+
+ return self._run_command(args, "Able to build without patch", "Unable to build without patch")
+
</ins><span class="cx"> def _test(self):
</span><del>- return self._run_command([
</del><ins>+ args = [
</ins><span class="cx"> "build-and-test",
</span><span class="cx"> "--no-clean",
</span><span class="cx"> "--no-update",
</span><span class="lines">@@ -151,12 +174,15 @@
</span><span class="cx"> "--test",
</span><span class="cx"> "--non-interactive",
</span><span class="cx"> "--build-style=%s" % self._delegate.build_style(),
</span><del>- ],
- "Passed tests",
- "Patch does not pass tests")
</del><ins>+ ]
</ins><span class="cx">
</span><ins>+ if hasattr(self._delegate, 'group'):
+ args.append("--group=%s" % self._delegate.group())
+
+ return self._run_command(args, "Passed tests", "Patch does not pass tests")
+
</ins><span class="cx"> def _build_and_test_without_patch(self):
</span><del>- return self._run_command([
</del><ins>+ args = [
</ins><span class="cx"> "build-and-test",
</span><span class="cx"> "--force-clean",
</span><span class="cx"> "--no-update",
</span><span class="lines">@@ -164,10 +190,13 @@
</span><span class="cx"> "--test",
</span><span class="cx"> "--non-interactive",
</span><span class="cx"> "--build-style=%s" % self._delegate.build_style(),
</span><del>- ],
- "Able to pass tests without patch",
- "Unable to pass tests without patch (tree is red?)")
</del><ins>+ ]
</ins><span class="cx">
</span><ins>+ if hasattr(self._delegate, 'group'):
+ args.append("--group=%s" % self._delegate.group())
+
+ return self._run_command(args, "Able to pass tests without patch", "Unable to pass tests without patch (tree is red?)")
+
</ins><span class="cx"> def _land(self):
</span><span class="cx"> # Unclear if this should pass --quiet or not. If --parent-command always does the reporting, then it should.
</span><span class="cx"> return self._run_command([
</span><span class="lines">@@ -206,10 +235,35 @@
</span><span class="cx"> # also present without the patch, so we don't need to defer.
</span><span class="cx"> return False
</span><span class="cx">
</span><del>- def _test_patch(self):
</del><ins>+ # FIXME: Abstract out common parts of the retry logic.
+ def _retry_jsc_tests(self):
+ first_results = self._delegate.test_results()
+ first_script_error = self._script_error
+ first_failure_status_id = self.failure_status_id
+ if first_results is None:
+ return False
+
</ins><span class="cx"> if self._test():
</span><span class="cx"> return True
</span><ins>+ second_results = self._delegate.test_results()
+ second_script_error = self._script_error
+ if second_results is None:
+ return False
</ins><span class="cx">
</span><ins>+ consistently_failing_test_results = JSCTestResults.intersection(first_results, second_results)
+
+ self._build_and_test_without_patch()
+ clean_tree_results = self._delegate.test_results()
+ if clean_tree_results is None:
+ return False
+
+ if consistently_failing_test_results.is_subset(clean_tree_results):
+ return True
+
+ self.failure_status_id = first_failure_status_id
+ return self.report_failure(None, consistently_failing_test_results, first_script_error)
+
+ def _retry_layout_tests(self):
</ins><span class="cx"> # Note: archive_last_test_results deletes the results directory, making these calls order-sensitve.
</span><span class="cx"> # We could remove this dependency by building the test_results from the archive.
</span><span class="cx"> first_results = self._delegate.test_results()
</span><span class="lines">@@ -240,11 +294,13 @@
</span><span class="cx"> return self.report_failure(first_results_archive, first_results, first_script_error)
</span><span class="cx">
</span><span class="cx"> if second_results.did_exceed_test_failure_limit():
</span><del>- self._should_defer_patch_or_throw(first_results.failing_test_results(), first_results_archive, first_script_error, first_failure_status_id)
</del><ins>+ self._should_defer_patch_or_throw(first_results.failing_test_results(), first_results_archive,
+ first_script_error, first_failure_status_id)
</ins><span class="cx"> return False
</span><span class="cx">
</span><span class="cx"> if first_results.did_exceed_test_failure_limit():
</span><del>- self._should_defer_patch_or_throw(second_results.failing_test_results(), second_results_archive, second_script_error, second_failure_status_id)
</del><ins>+ self._should_defer_patch_or_throw(second_results.failing_test_results(), second_results_archive,
+ second_script_error, second_failure_status_id)
</ins><span class="cx"> return False
</span><span class="cx">
</span><span class="cx"> if self._results_failed_different_tests(first_results, second_results):
</span><span class="lines">@@ -259,7 +315,8 @@
</span><span class="cx">
</span><span class="cx"> tests_that_consistently_failed = first_failing_results_set.intersection(second_failing_results_set)
</span><span class="cx"> if tests_that_consistently_failed:
</span><del>- if self._should_defer_patch_or_throw(tests_that_consistently_failed, first_results_archive, first_script_error, first_failure_status_id):
</del><ins>+ if self._should_defer_patch_or_throw(tests_that_consistently_failed, first_results_archive,
+ first_script_error, first_failure_status_id):
</ins><span class="cx"> return False # Defer patch
</span><span class="cx">
</span><span class="cx"> # At this point we know that at least one test flaked, but no consistent failures
</span><span class="lines">@@ -266,7 +323,8 @@
</span><span class="cx"> # were introduced. This is a bit of a grey-zone.
</span><span class="cx"> return False # Defer patch
</span><span class="cx">
</span><del>- if self._should_defer_patch_or_throw(first_results.failing_test_results(), first_results_archive, first_script_error, first_failure_status_id):
</del><ins>+ if self._should_defer_patch_or_throw(first_results.failing_test_results(), first_results_archive,
+ first_script_error, first_failure_status_id):
</ins><span class="cx"> return False # Defer patch
</span><span class="cx">
</span><span class="cx"> # At this point, we know that the first and second runs had the exact same failures,
</span><span class="lines">@@ -274,6 +332,15 @@
</span><span class="cx"> # that the patch is good.
</span><span class="cx"> return True
</span><span class="cx">
</span><ins>+ def _test_patch(self):
+ if self._test():
+ return True
+
+ if hasattr(self._delegate, 'group') and self._delegate.group() == "jsc":
+ return self._retry_jsc_tests()
+ else:
+ return self._retry_layout_tests()
+
</ins><span class="cx"> def results_archive_from_patch_test_run(self, patch):
</span><span class="cx"> assert(self._patch.id() == patch.id()) # PatchAnalysisTask is not currently re-useable.
</span><span class="cx"> return self._results_archive_from_patch_test_run
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpytoolcommandsdownloadpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/tool/commands/download.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/tool/commands/download.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/tool/commands/download.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> # Copyright (c) 2009, 2011 Google Inc. All rights reserved.
</span><del>-# Copyright (c) 2009 Apple Inc. All rights reserved.
</del><ins>+# Copyright (c) 2009, 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -87,6 +87,14 @@
</span><span class="cx"> ]
</span><span class="cx">
</span><span class="cx">
</span><ins>+class CheckPatchRelevance(AbstractSequencedCommand):
+ name = "check-patch-relevance"
+ help_text = "Check if this patch needs to be tested"
+ steps = [
+ steps.CheckPatchRelevance,
+ ]
+
+
</ins><span class="cx"> class Land(AbstractSequencedCommand):
</span><span class="cx"> name = "land"
</span><span class="cx"> help_text = "Land the current working directory diff and updates the associated bug if any"
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpytoolcommandsdownload_unittestpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/tool/commands/download_unittest.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/tool/commands/download_unittest.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/tool/commands/download_unittest.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -89,6 +89,7 @@
</span><span class="cx"> options.update = True
</span><span class="cx"> options.architecture = 'MOCK ARCH'
</span><span class="cx"> options.iterate_on_new_tests = 0
</span><ins>+ options.group = None
</ins><span class="cx"> return options
</span><span class="cx">
</span><span class="cx"> def test_build(self):
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpytoolcommandsearlywarningsystempy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/tool/commands/earlywarningsystem.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/tool/commands/earlywarningsystem.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/tool/commands/earlywarningsystem.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,4 +1,5 @@
</span><span class="cx"> # Copyright (c) 2009 Google Inc. All rights reserved.
</span><ins>+# Copyright (c) 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -28,6 +29,7 @@
</span><span class="cx">
</span><span class="cx"> import json
</span><span class="cx"> import logging
</span><ins>+import os
</ins><span class="cx"> from optparse import make_option
</span><span class="cx">
</span><span class="cx"> from webkitpy.common.config.committers import CommitterList
</span><span class="lines">@@ -36,7 +38,8 @@
</span><span class="cx"> from webkitpy.common.system.executive import ScriptError
</span><span class="cx"> from webkitpy.tool.bot.earlywarningsystemtask import EarlyWarningSystemTask, EarlyWarningSystemTaskDelegate
</span><span class="cx"> from webkitpy.tool.bot.layouttestresultsreader import LayoutTestResultsReader
</span><del>-from webkitpy.tool.bot.patchanalysistask import UnableToApplyPatch, PatchIsNotValid
</del><ins>+from webkitpy.tool.bot.jsctestresultsreader import JSCTestResultsReader
+from webkitpy.tool.bot.patchanalysistask import UnableToApplyPatch, PatchIsNotValid, PatchIsNotApplicable
</ins><span class="cx"> from webkitpy.tool.bot.queueengine import QueueEngine
</span><span class="cx"> from webkitpy.tool.commands.queues import AbstractReviewQueue
</span><span class="cx">
</span><span class="lines">@@ -53,8 +56,12 @@
</span><span class="cx">
</span><span class="cx"> def begin_work_queue(self):
</span><span class="cx"> AbstractReviewQueue.begin_work_queue(self)
</span><del>- self._layout_test_results_reader = LayoutTestResultsReader(self._tool, self._port.results_directory(), self._log_directory())
</del><span class="cx">
</span><ins>+ if self.group() == "jsc":
+ self._test_results_reader = JSCTestResultsReader(self._tool, self._port.jsc_results_directory())
+ else:
+ self._test_results_reader = LayoutTestResultsReader(self._tool, self._port.results_directory(), self._log_directory())
+
</ins><span class="cx"> def _failing_tests_message(self, task, patch):
</span><span class="cx"> results = task.results_from_patch_test_run(patch)
</span><span class="cx">
</span><span class="lines">@@ -79,8 +86,12 @@
</span><span class="cx"> tool.bugs.add_cc_to_bug(patch.bug_id(), self.watchers)
</span><span class="cx"> tool.bugs.set_flag_on_attachment(patch.id(), "commit-queue", "-", message)
</span><span class="cx">
</span><ins>+ # This exists for mocking
+ def _create_task(self, patch):
+ return EarlyWarningSystemTask(self, patch, self._options.run_tests)
+
</ins><span class="cx"> def review_patch(self, patch):
</span><del>- task = EarlyWarningSystemTask(self, patch, self._options.run_tests)
</del><ins>+ task = self._create_task(patch)
</ins><span class="cx"> try:
</span><span class="cx"> succeeded = task.run()
</span><span class="cx"> if not succeeded:
</span><span class="lines">@@ -93,6 +104,9 @@
</span><span class="cx"> except UnableToApplyPatch, e:
</span><span class="cx"> self._did_error(patch, "%s unable to apply patch." % self.name)
</span><span class="cx"> return False
</span><ins>+ except PatchIsNotApplicable, e:
+ self._did_skip(patch)
+ return False
</ins><span class="cx"> except ScriptError, e:
</span><span class="cx"> self._post_reject_message_on_bug(self._tool, patch, task.failure_status_id, self._failing_tests_message(task, patch))
</span><span class="cx"> results_archive = task.results_archive_from_patch_test_run(patch)
</span><span class="lines">@@ -117,14 +131,17 @@
</span><span class="cx"> return self._update_status(message, patch=patch, results_file=failure_log)
</span><span class="cx">
</span><span class="cx"> def test_results(self):
</span><del>- return self._layout_test_results_reader.results()
</del><ins>+ return self._test_results_reader.results()
</ins><span class="cx">
</span><span class="cx"> def archive_last_test_results(self, patch):
</span><del>- return self._layout_test_results_reader.archive(patch)
</del><ins>+ return self._test_results_reader.archive(patch)
</ins><span class="cx">
</span><span class="cx"> def build_style(self):
</span><span class="cx"> return self._build_style
</span><span class="cx">
</span><ins>+ def group(self):
+ return self._group
+
</ins><span class="cx"> def refetch_patch(self, patch):
</span><span class="cx"> return self._tool.bugs.fetch_attachment(patch.id())
</span><span class="cx">
</span><span class="lines">@@ -149,7 +166,7 @@
</span><span class="cx">
</span><span class="cx"> classes = []
</span><span class="cx"> for name, config in ewses.iteritems():
</span><del>- classes.append(type(name.encode('utf-8').translate(None, ' -'), (AbstractEarlyWarningSystem,), {
</del><ins>+ classes.append(type(name.encode('utf-8').translate(None, ' -'), (cls,), {
</ins><span class="cx"> 'name': config.get('name', config['port'] + '-ews'),
</span><span class="cx"> 'port_name': config['port'],
</span><span class="cx"> 'architecture': config.get('architecture', None),
</span><span class="lines">@@ -156,5 +173,6 @@
</span><span class="cx"> '_build_style': config.get('style', "release"),
</span><span class="cx"> 'watchers': config.get('watchers', []),
</span><span class="cx"> 'run_tests': config.get('runTests', cls.run_tests),
</span><ins>+ '_group': config.get('group', None),
</ins><span class="cx"> }))
</span><span class="cx"> return classes
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpytoolcommandsearlywarningsystem_unittestpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/tool/commands/earlywarningsystem_unittest.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/tool/commands/earlywarningsystem_unittest.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/tool/commands/earlywarningsystem_unittest.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,4 +1,5 @@
</span><span class="cx"> # Copyright (C) 2009 Google Inc. All rights reserved.
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -29,6 +30,7 @@
</span><span class="cx"> from webkitpy.thirdparty.mock import Mock
</span><span class="cx"> from webkitpy.common.host import Host
</span><span class="cx"> from webkitpy.common.host_mock import MockHost
</span><ins>+from webkitpy.common.net.jsctestresults import JSCTestResults
</ins><span class="cx"> from webkitpy.common.net.layouttestresults import LayoutTestResults
</span><span class="cx"> from webkitpy.common.system.outputcapture import OutputCapture
</span><span class="cx"> from webkitpy.layout_tests.models import test_results
</span><span class="lines">@@ -41,40 +43,79 @@
</span><span class="cx"> from webkitpy.tool.mocktool import MockTool, MockOptions
</span><span class="cx">
</span><span class="cx">
</span><ins>+# Needed to define port_name, used in AbstractEarlyWarningSystem.__init__
+class TestEWS(AbstractEarlyWarningSystem):
+ port_name = "win" # Needs to be a port which port/factory understands.
+ _build_style = None
+ _group = None
+
+
+class TestJSCEWS(AbstractEarlyWarningSystem):
+ port_name = "mac" # Needs to be a port which port/factory understands.
+ _build_style = None
+ _group = "jsc"
+
+
</ins><span class="cx"> class AbstractEarlyWarningSystemTest(QueuesTest):
</span><del>- def test_failing_tests_message(self):
- # Needed to define port_name, used in AbstractEarlyWarningSystem.__init__
- class TestEWS(AbstractEarlyWarningSystem):
- port_name = "win" # Needs to be a port which port/factory understands.
- _build_style = None
-
- ews = TestEWS()
</del><ins>+ def _test_message(self, ews, results, message):
</ins><span class="cx"> ews.bind_to_tool(MockTool())
</span><span class="cx"> ews.host = MockHost()
</span><span class="cx"> ews._options = MockOptions(port=None, confirm=False)
</span><span class="cx"> OutputCapture().assert_outputs(self, ews.begin_work_queue, expected_logs=self._default_begin_work_queue_logs(ews.name))
</span><span class="cx"> task = Mock()
</span><del>- task.results_from_patch_test_run = lambda a: LayoutTestResults([test_results.TestResult("foo.html", failures=[test_failures.FailureTextMismatch()]),
- test_results.TestResult("bar.html", failures=[test_failures.FailureTextMismatch()])],
- did_exceed_test_failure_limit=False)
</del><ins>+ task.results_from_patch_test_run = results
</ins><span class="cx"> patch = ews._tool.bugs.fetch_attachment(10000)
</span><del>- self.assertMultiLineEqual(ews._failing_tests_message(task, patch), "New failing tests:\nfoo.html\nbar.html")
</del><ins>+ self.assertMultiLineEqual(ews._failing_tests_message(task, patch), message)
</ins><span class="cx">
</span><ins>+ def test_failing_tests_message(self):
+ ews = TestEWS()
+ results = lambda a: LayoutTestResults([test_results.TestResult("foo.html", failures=[test_failures.FailureTextMismatch()]),
+ test_results.TestResult("bar.html", failures=[test_failures.FailureTextMismatch()])],
+ did_exceed_test_failure_limit=False)
+ message = "New failing tests:\nfoo.html\nbar.html"
+ self._test_message(ews, results, message)
</ins><span class="cx">
</span><ins>+ def test_failing_jsc_tests_message(self):
+ ews = TestJSCEWS()
+ results = lambda a: JSCTestResults(False, ["es6.yaml/es6/typed_arrays_Int8Array.js.default", "es6.yaml/es6/typed_arrays_Uint8Array.js.default"])
+ message = "New failing tests:\nes6.yaml/es6/typed_arrays_Int8Array.js.default\nes6.yaml/es6/typed_arrays_Uint8Array.js.default\napiTests"
+ self._test_message(ews, results, message)
+
+
+class MockEarlyWarningSystemTaskForInconclusiveJSCResults(EarlyWarningSystemTask):
+ def _test_patch(self):
+ self._test()
+ results = self._delegate.test_results()
+ return bool(results)
+
+
+class MockAbstractEarlyWarningSystemForInconclusiveJSCResults(AbstractEarlyWarningSystem):
+ def _create_task(self, patch):
+ task = MockEarlyWarningSystemTaskForInconclusiveJSCResults(self, patch, self._options.run_tests)
+ return task
+
+
</ins><span class="cx"> class EarlyWarningSystemTest(QueuesTest):
</span><del>- def _default_expected_logs(self, ews):
</del><ins>+ def _default_expected_logs(self, ews, conclusive):
</ins><span class="cx"> string_replacements = {
</span><span class="cx"> "name": ews.name,
</span><span class="cx"> "port": ews.port_name,
</span><span class="cx"> "architecture": " --architecture=%s" % ews.architecture if ews.architecture else "",
</span><span class="cx"> "build_style": ews.build_style(),
</span><ins>+ "group": ews.group(),
</ins><span class="cx"> }
</span><span class="cx"> if ews.run_tests:
</span><del>- run_tests_line = "Running: webkit-patch --status-host=example.com build-and-test --no-clean --no-update --test --non-interactive --build-style=%(build_style)s --port=%(port)s%(architecture)s\n" % string_replacements
</del><ins>+ run_tests_line = "Running: webkit-patch --status-host=example.com build-and-test --no-clean --no-update --test --non-interactive --build-style=%(build_style)s --group=%(group)s --port=%(port)s%(architecture)s\n" % string_replacements
</ins><span class="cx"> else:
</span><span class="cx"> run_tests_line = ""
</span><span class="cx"> string_replacements['run_tests_line'] = run_tests_line
</span><span class="cx">
</span><ins>+ if conclusive:
+ result_lines = "MOCK: update_status: %(name)s Pass\nMOCK: release_work_item: %(name)s 10000\n" % string_replacements
+ else:
+ result_lines = "MOCK: release_lock: %(name)s 10000\n" % string_replacements
+ string_replacements['result_lines'] = result_lines
+
</ins><span class="cx"> expected_logs = {
</span><span class="cx"> "begin_work_queue": self._default_begin_work_queue_logs(ews.name),
</span><span class="cx"> "process_work_item": """MOCK: update_status: %(name)s Started processing patch
</span><span class="lines">@@ -81,22 +122,21 @@
</span><span class="cx"> Running: webkit-patch --status-host=example.com clean --port=%(port)s%(architecture)s
</span><span class="cx"> Running: webkit-patch --status-host=example.com update --port=%(port)s%(architecture)s
</span><span class="cx"> Running: webkit-patch --status-host=example.com apply-attachment --no-update --non-interactive 10000 --port=%(port)s%(architecture)s
</span><del>-Running: webkit-patch --status-host=example.com build --no-clean --no-update --build-style=%(build_style)s --port=%(port)s%(architecture)s
-%(run_tests_line)sMOCK: update_status: %(name)s Pass
-MOCK: release_work_item: %(name)s 10000
-""" % string_replacements,
</del><ins>+Running: webkit-patch --status-host=example.com check-patch-relevance --group=%(group)s --port=%(port)s%(architecture)s
+Running: webkit-patch --status-host=example.com build --no-clean --no-update --build-style=%(build_style)s --group=%(group)s --port=%(port)s%(architecture)s
+%(run_tests_line)s%(result_lines)s""" % string_replacements,
</ins><span class="cx"> "handle_unexpected_error": "Mock error message\n",
</span><span class="cx"> "handle_script_error": "ScriptError error message\n\nMOCK output\n",
</span><span class="cx"> }
</span><span class="cx"> return expected_logs
</span><span class="cx">
</span><del>- def _test_ews(self, ews):
</del><ins>+ def _test_ews(self, ews, results_are_conclusive=True):
</ins><span class="cx"> ews.bind_to_tool(MockTool())
</span><span class="cx"> ews.host = MockHost()
</span><span class="cx"> options = Mock()
</span><span class="cx"> options.port = None
</span><span class="cx"> options.run_tests = ews.run_tests
</span><del>- self.assert_queue_outputs(ews, expected_logs=self._default_expected_logs(ews), options=options)
</del><ins>+ self.assert_queue_outputs(ews, expected_logs=self._default_expected_logs(ews, results_are_conclusive), options=options)
</ins><span class="cx">
</span><span class="cx"> def test_ewses(self):
</span><span class="cx"> classes = AbstractEarlyWarningSystem.load_ews_classes()
</span><span class="lines">@@ -104,3 +144,11 @@
</span><span class="cx"> self.maxDiff = None
</span><span class="cx"> for ews_class in classes:
</span><span class="cx"> self._test_ews(ews_class())
</span><ins>+
+ def test_inconclusive_jsc_test_results(self):
+ classes = MockAbstractEarlyWarningSystemForInconclusiveJSCResults.load_ews_classes()
+ self.assertTrue(classes)
+ self.maxDiff = None
+ test_classes = filter(lambda x: x.run_tests and x.group == "jsc", classes)
+ for ews_class in test_classes:
+ self._test_ews(ews_class(), False)
</ins></span></pre></div>
<a id="trunkToolsScriptswebkitpytoolcommandsqueuespy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/tool/commands/queues.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/tool/commands/queues.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/tool/commands/queues.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,5 +1,5 @@
</span><span class="cx"> # Copyright (c) 2009 Google Inc. All rights reserved.
</span><del>-# Copyright (c) 2009 Apple Inc. All rights reserved.
</del><ins>+# Copyright (c) 2009, 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -62,6 +62,7 @@
</span><span class="cx"> watchers = [
</span><span class="cx"> ]
</span><span class="cx">
</span><ins>+ _skip_status = "Skip"
</ins><span class="cx"> _pass_status = "Pass"
</span><span class="cx"> _fail_status = "Fail"
</span><span class="cx"> _error_status = "Error"
</span><span class="lines">@@ -244,6 +245,10 @@
</span><span class="cx"> self._update_status(message, patch)
</span><span class="cx"> self._release_work_item(patch)
</span><span class="cx">
</span><ins>+ def _did_skip(self, patch):
+ self._update_status(self._skip_status, patch)
+ self._release_work_item(patch)
+
</ins><span class="cx"> def _unlock_patch(self, patch):
</span><span class="cx"> self._tool.status_server.release_lock(self.name, patch)
</span><span class="cx">
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpytoolsteps__init__py"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/tool/steps/__init__.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/tool/steps/__init__.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/tool/steps/__init__.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,4 +1,5 @@
</span><span class="cx"> # Copyright (C) 2010 Google Inc. All rights reserved.
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -33,6 +34,7 @@
</span><span class="cx"> from webkitpy.tool.steps.applywatchlist import ApplyWatchList
</span><span class="cx"> from webkitpy.tool.steps.attachtobug import AttachToBug
</span><span class="cx"> from webkitpy.tool.steps.build import Build
</span><ins>+from webkitpy.tool.steps.checkpatchrelevance import CheckPatchRelevance
</ins><span class="cx"> from webkitpy.tool.steps.checkstyle import CheckStyle
</span><span class="cx"> from webkitpy.tool.steps.cleanworkingdirectory import CleanWorkingDirectory
</span><span class="cx"> from webkitpy.tool.steps.closebug import CloseBug
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpytoolstepsbuildpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/tool/steps/build.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/tool/steps/build.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/tool/steps/build.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,4 +1,5 @@
</span><span class="cx"> # Copyright (C) 2010 Google Inc. All rights reserved.
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -41,20 +42,24 @@
</span><span class="cx"> Options.build,
</span><span class="cx"> Options.quiet,
</span><span class="cx"> Options.build_style,
</span><ins>+ Options.group,
</ins><span class="cx"> ]
</span><span class="cx">
</span><del>- def build(self, build_style):
</del><ins>+ def build(self, build_style, group):
</ins><span class="cx"> environment = self._tool.copy_current_environment()
</span><span class="cx"> environment.disable_gcc_smartquotes()
</span><span class="cx"> environment.disable_jhbuild_VT100_output()
</span><span class="cx"> env = environment.to_dictionary()
</span><span class="cx">
</span><del>- build_webkit_command = self._tool.deprecated_port().build_webkit_command(build_style=build_style)
</del><ins>+ if group == "jsc":
+ build_command = self._tool.deprecated_port().build_jsc_command(build_style=build_style)
+ else:
+ build_command = self._tool.deprecated_port().build_webkit_command(build_style=build_style)
</ins><span class="cx">
</span><span class="cx"> if self._options.architecture:
</span><del>- build_webkit_command += ['ARCHS=%s' % self._options.architecture]
</del><ins>+ build_command += ['ARCHS=%s' % self._options.architecture]
</ins><span class="cx">
</span><del>- self._tool.executive.run_and_throw_if_fail(build_webkit_command, self._options.quiet,
</del><ins>+ self._tool.executive.run_and_throw_if_fail(build_command, self._options.quiet,
</ins><span class="cx"> cwd=self._tool.scm().checkout_root, env=env)
</span><span class="cx">
</span><span class="cx"> def run(self, state):
</span><span class="lines">@@ -61,8 +66,11 @@
</span><span class="cx"> if not self._options.build:
</span><span class="cx"> return
</span><span class="cx"> _log.info("Building WebKit")
</span><ins>+
+ group = self._options.group
+
</ins><span class="cx"> if self._options.build_style == "both":
</span><del>- self.build("debug")
- self.build("release")
</del><ins>+ self.build("debug", group)
+ self.build("release", group)
</ins><span class="cx"> else:
</span><del>- self.build(self._options.build_style)
</del><ins>+ self.build(self._options.build_style, group)
</ins></span></pre></div>
<a id="trunkToolsScriptswebkitpytoolstepscheckpatchrelevancepy"></a>
<div class="addfile"><h4>Added: trunk/Tools/Scripts/webkitpy/tool/steps/checkpatchrelevance.py (0 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/tool/steps/checkpatchrelevance.py (rev 0)
+++ trunk/Tools/Scripts/webkitpy/tool/steps/checkpatchrelevance.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -0,0 +1,74 @@
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' AND
+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS BE LIABLE FOR
+# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import logging
+import re
+
+from webkitpy.common.system.executive import ScriptError
+from webkitpy.tool.steps.abstractstep import AbstractStep
+from webkitpy.tool.steps.options import Options
+
+_log = logging.getLogger(__name__)
+
+
+class CheckPatchRelevance(AbstractStep):
+ @classmethod
+ def options(cls):
+ return AbstractStep.options() + [
+ Options.group,
+ ]
+
+ jsc_paths = [
+ "JSTests/",
+ "Source/JavaScriptCore/",
+ "Source/WTF/"
+ "Source/bmalloc/",
+ ]
+
+ group_to_paths_mapping = {
+ 'jsc': jsc_paths,
+ }
+
+ def _changes_are_relevant(self, changed_files):
+ # In the default case, all patches are relevant
+ if self._options.group is None:
+ return True
+
+ patterns = self.group_to_paths_mapping[self._options.group]
+
+ for changed_file in changed_files:
+ for pattern in patterns:
+ if re.search(pattern, changed_file, re.IGNORECASE):
+ return True
+
+ return False
+
+ def run(self, state):
+ _log.info("Checking relevance of patch")
+
+ change_list = self._tool.scm().changed_files()
+
+ if self._changes_are_relevant(change_list):
+ return True
+
+ _log.info("This patch does not have relevant changes.")
+ raise ScriptError(message="This patch does not have relevant changes.")
</ins></span></pre></div>
<a id="trunkToolsScriptswebkitpytoolstepsoptionspy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/tool/steps/options.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/tool/steps/options.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/tool/steps/options.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,4 +1,5 @@
</span><span class="cx"> # Copyright (C) 2010 Google Inc. All rights reserved.
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -46,6 +47,7 @@
</span><span class="cx"> email = make_option("--email", action="store", type="string", dest="email", help="Email address to use in ChangeLogs.")
</span><span class="cx"> force_clean = make_option("--force-clean", action="store_true", dest="force_clean", default=False, help="Clean working directory before applying patches (removes local changes and commits)")
</span><span class="cx"> git_commit = make_option("-g", "--git-commit", action="store", dest="git_commit", help="Operate on a local commit. If a range, the commits are squashed into one. <ref>.... includes the working copy changes. UPSTREAM can be used for the upstream/tracking branch.")
</span><ins>+ group = make_option("--group", action="store", dest="group", default=None, help="")
</ins><span class="cx"> local_commit = make_option("--local-commit", action="store_true", dest="local_commit", default=False, help="Make a local commit for each applied patch")
</span><span class="cx"> non_interactive = make_option("--non-interactive", action="store_true", dest="non_interactive", default=False, help="Never prompt the user, fail as fast as possible.")
</span><span class="cx"> obsolete_patches = make_option("--no-obsolete", action="store_false", dest="obsolete_patches", default=True, help="Do not obsolete old patches before posting this one.")
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpytoolstepsruntestspy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/tool/steps/runtests.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/tool/steps/runtests.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/tool/steps/runtests.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,4 +1,5 @@
</span><span class="cx"> # Copyright (C) 2010 Google Inc. All rights reserved.
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -53,9 +54,14 @@
</span><span class="cx"> Options.iterate_on_new_tests,
</span><span class="cx"> Options.non_interactive,
</span><span class="cx"> Options.quiet,
</span><ins>+ Options.group,
</ins><span class="cx"> ]
</span><span class="cx">
</span><span class="cx"> def run(self, state):
</span><ins>+ if self._options.group == "jsc":
+ self._run_javascriptcore_tests()
+ return
+
</ins><span class="cx"> if self._options.iterate_on_new_tests:
</span><span class="cx"> _log.info("Running run-webkit-tests on new tests")
</span><span class="cx"> self._run_webkit_tests(self._options.iterate_on_new_tests)
</span><span class="lines">@@ -143,3 +149,11 @@
</span><span class="cx"> args.append("--iterations=%d" % iterate_on_new_tests)
</span><span class="cx">
</span><span class="cx"> self._tool.executive.run_and_throw_if_fail(args, cwd=self._tool.scm().checkout_root)
</span><ins>+
+ def _run_javascriptcore_tests(self):
+ args = self._tool.deprecated_port().run_javascriptcore_tests_command(self._options.build_style)
+
+ results_directory = self._tool.port_factory.get().jsc_results_directory()
+ results_file_path = self._tool.filesystem.join(results_directory, "jsc_test_results.json")
+ args.append("--json-output=%s" % results_file_path)
+ self._tool.executive.run_and_throw_if_fail(args, cwd=self._tool.scm().checkout_root)
</ins></span></pre></div>
<a id="trunkToolsScriptswebkitpytoolstepsruntests_unittestpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/tool/steps/runtests_unittest.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/tool/steps/runtests_unittest.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/tool/steps/runtests_unittest.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -40,7 +40,7 @@
</span><span class="cx"> tool = MockTool(log_executive=True)
</span><span class="cx"> tool._deprecated_port.run_python_unittests_command = lambda: None
</span><span class="cx"> tool._deprecated_port.run_perl_unittests_command = lambda: None
</span><del>- step = RunTests(tool, MockOptions(test=True, non_interactive=True, quiet=False, build_style="release", iterate_on_new_tests=0))
</del><ins>+ step = RunTests(tool, MockOptions(test=True, non_interactive=True, quiet=False, build_style="release", iterate_on_new_tests=0, group=None))
</ins><span class="cx">
</span><span class="cx"> if sys.platform != "cygwin":
</span><span class="cx"> expected_logs = """Running bindings generation tests
</span></span></pre></div>
<a id="trunkToolsScriptswebkitpytoolstepssteps_unittestpy"></a>
<div class="modfile"><h4>Modified: trunk/Tools/Scripts/webkitpy/tool/steps/steps_unittest.py (212578 => 212579)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/Scripts/webkitpy/tool/steps/steps_unittest.py 2017-02-17 22:39:45 UTC (rev 212578)
+++ trunk/Tools/Scripts/webkitpy/tool/steps/steps_unittest.py 2017-02-17 22:41:49 UTC (rev 212579)
</span><span class="lines">@@ -1,4 +1,5 @@
</span><span class="cx"> # Copyright (C) 2010 Google Inc. All rights reserved.
</span><ins>+# Copyright (C) 2017 Apple Inc. All rights reserved.
</ins><span class="cx"> #
</span><span class="cx"> # Redistribution and use in source and binary forms, with or without
</span><span class="cx"> # modification, are permitted provided that the following conditions are
</span><span class="lines">@@ -28,6 +29,7 @@
</span><span class="cx">
</span><span class="cx"> import unittest
</span><span class="cx">
</span><ins>+from webkitpy.common.system.executive import ScriptError
</ins><span class="cx"> from webkitpy.common.system.outputcapture import OutputCapture
</span><span class="cx"> from webkitpy.common.config.ports import DeprecatedPort
</span><span class="cx"> from webkitpy.tool.mocktool import MockOptions, MockTool
</span><span class="lines">@@ -38,6 +40,7 @@
</span><span class="cx"> class StepsTest(unittest.TestCase):
</span><span class="cx"> def _step_options(self):
</span><span class="cx"> options = MockOptions()
</span><ins>+ options.group = None
</ins><span class="cx"> options.non_interactive = True
</span><span class="cx"> options.port = 'MOCK port'
</span><span class="cx"> options.quiet = True
</span><span class="lines">@@ -111,7 +114,7 @@
</span><span class="cx"> Running Perl unit tests
</span><span class="cx"> MOCK run_and_throw_if_fail: ['Tools/Scripts/test-webkitperl'], cwd=/mock-checkout
</span><span class="cx"> Running JavaScriptCore tests
</span><del>-MOCK run_and_throw_if_fail: ['Tools/Scripts/run-javascriptcore-tests'], cwd=/mock-checkout
</del><ins>+MOCK run_and_throw_if_fail: ['Tools/Scripts/run-javascriptcore-tests', '--no-fail-fast'], cwd=/mock-checkout
</ins><span class="cx"> Running bindings generation tests
</span><span class="cx"> MOCK run_and_throw_if_fail: ['Tools/Scripts/run-bindings-tests'], cwd=/mock-checkout
</span><span class="cx"> Running run-webkit-tests
</span><span class="lines">@@ -133,7 +136,7 @@
</span><span class="cx"> Running Perl unit tests
</span><span class="cx"> MOCK run_and_throw_if_fail: ['Tools/Scripts/test-webkitperl'], cwd=/mock-checkout
</span><span class="cx"> Running JavaScriptCore tests
</span><del>-MOCK run_and_throw_if_fail: ['Tools/Scripts/run-javascriptcore-tests'], cwd=/mock-checkout
</del><ins>+MOCK run_and_throw_if_fail: ['Tools/Scripts/run-javascriptcore-tests', '--no-fail-fast'], cwd=/mock-checkout
</ins><span class="cx"> Running bindings generation tests
</span><span class="cx"> MOCK run_and_throw_if_fail: ['Tools/Scripts/run-bindings-tests'], cwd=/mock-checkout
</span><span class="cx"> Running run-webkit-tests
</span><span class="lines">@@ -140,3 +143,104 @@
</span><span class="cx"> MOCK run_and_throw_if_fail: ['Tools/Scripts/run-webkit-tests', '--debug', '--quiet'], cwd=/mock-checkout
</span><span class="cx"> """
</span><span class="cx"> OutputCapture().assert_outputs(self, step.run, [{}], expected_logs=expected_logs)
</span><ins>+
+ def test_runtests_jsc(self):
+ mock_options = self._step_options()
+ mock_options.non_interactive = False
+ mock_options.build_style = "release"
+ mock_options.group = "jsc"
+ step = steps.RunTests(MockTool(log_executive=True), mock_options)
+ tool = MockTool(log_executive=True)
+ # FIXME: We shouldn't use a real port-object here, but there is too much to mock at the moment.
+ tool._deprecated_port = DeprecatedPort()
+ step = steps.RunTests(tool, mock_options)
+ expected_logs = """MOCK run_command: ['perl', 'Tools/Scripts/webkit-build-directory', '--configuration', '--release', '--mac'], cwd=/mock-checkout
+MOCK run_and_throw_if_fail: ['Tools/Scripts/run-javascriptcore-tests', '--no-fail-fast', '--release', '--json-output=/MOCK output of child process/jsc_test_results.json'], cwd=/mock-checkout
+"""
+ OutputCapture().assert_outputs(self, step.run, [{}], expected_logs=expected_logs)
+
+ def test_runtests_jsc_debug(self):
+ mock_options = self._step_options()
+ mock_options.non_interactive = False
+ mock_options.build_style = "debug"
+ mock_options.group = "jsc"
+ tool = MockTool(log_executive=True)
+ # FIXME: We shouldn't use a real port-object here, but there is too much to mock at the moment.
+ tool._deprecated_port = DeprecatedPort()
+ step = steps.RunTests(tool, mock_options)
+ expected_logs = """MOCK run_command: ['perl', 'Tools/Scripts/webkit-build-directory', '--configuration', '--release', '--mac'], cwd=/mock-checkout
+MOCK run_and_throw_if_fail: ['Tools/Scripts/run-javascriptcore-tests', '--no-fail-fast', '--debug', '--json-output=/MOCK output of child process/jsc_test_results.json'], cwd=/mock-checkout
+"""
+ OutputCapture().assert_outputs(self, step.run, [{}], expected_logs=expected_logs)
+
+ def test_build_jsc_debug(self):
+ mock_options = self._step_options()
+ mock_options.non_interactive = False
+ mock_options.build_style = "debug"
+ mock_options.build = True
+ mock_options.architecture = True
+ mock_options.group = "jsc"
+ tool = MockTool(log_executive=True)
+ # FIXME: We shouldn't use a real port-object here, but there is too much to mock at the moment.
+ tool._deprecated_port = DeprecatedPort()
+ step = steps.Build(tool, mock_options)
+ expected_logs = """Building WebKit
+MOCK run_and_throw_if_fail: ['Tools/Scripts/build-jsc', '--debug', 'ARCHS=True'], cwd=/mock-checkout, env={'LC_ALL': 'C', 'TERM': 'none', 'MOCK_ENVIRON_COPY': '1'}
+"""
+ OutputCapture().assert_outputs(self, step.run, [{}], expected_logs=expected_logs)
+
+ def test_build_jsc(self):
+ mock_options = self._step_options()
+ mock_options.non_interactive = False
+ mock_options.build_style = "release"
+ mock_options.build = True
+ mock_options.architecture = True
+ mock_options.group = "jsc"
+ tool = MockTool(log_executive=True)
+ # FIXME: We shouldn't use a real port-object here, but there is too much to mock at the moment.
+ tool._deprecated_port = DeprecatedPort()
+ step = steps.Build(tool, mock_options)
+ expected_logs = """Building WebKit
+MOCK run_and_throw_if_fail: ['Tools/Scripts/build-jsc', '--release', 'ARCHS=True'], cwd=/mock-checkout, env={'LC_ALL': 'C', 'TERM': 'none', 'MOCK_ENVIRON_COPY': '1'}
+"""
+ OutputCapture().assert_outputs(self, step.run, [{}], expected_logs=expected_logs)
+
+ def test_patch_relevant(self):
+ self.maxDiff = None
+ mock_options = self._step_options()
+ tool = MockTool(log_executive=True)
+ tool.scm()._mockChangedFiles = ["JSTests/MockFile1", "ChangeLog"]
+ # FIXME: We shouldn't use a real port-object here, but there is too much to mock at the moment.
+ tool._deprecated_port = DeprecatedPort()
+ step = steps.CheckPatchRelevance(tool, mock_options)
+ expected_logs = """Checking relevance of patch
+"""
+ OutputCapture().assert_outputs(self, step.run, [{}], expected_logs=expected_logs)
+
+ def test_patch_relevant_jsc(self):
+ self.maxDiff = None
+ mock_options = self._step_options()
+ mock_options.group = "jsc"
+ tool = MockTool(log_executive=True)
+ tool.scm()._mockChangedFiles = ["JSTests/MockFile1", "ChangeLog"]
+ # FIXME: We shouldn't use a real port-object here, but there is too much to mock at the moment.
+ tool._deprecated_port = DeprecatedPort()
+ step = steps.CheckPatchRelevance(tool, mock_options)
+ expected_logs = """Checking relevance of patch
+"""
+ OutputCapture().assert_outputs(self, step.run, [{}], expected_logs=expected_logs)
+
+ def test_patch_not_relevant_jsc(self):
+ self.maxDiff = None
+ mock_options = self._step_options()
+ mock_options.group = "jsc"
+ tool = MockTool(log_executive=True)
+ tool.scm()._mockChangedFiles = ["Tools/ChangeLog", "Tools/Scripts/webkitpy/tool/steps/steps_unittest.py"]
+ # FIXME: We shouldn't use a real port-object here, but there is too much to mock at the moment.
+ tool._deprecated_port = DeprecatedPort()
+ step = steps.CheckPatchRelevance(tool, mock_options)
+ expected_logs = """Checking relevance of patch
+This patch does not have relevant changes.
+"""
+ with self.assertRaises(ScriptError):
+ OutputCapture().assert_outputs(self, step.run, [{}], expected_logs=expected_logs)
</ins></span></pre>
</div>
</div>
</body>
</html>