<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[189213] trunk</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/189213">189213</a></dd>
<dt>Author</dt> <dd>bburg@apple.com</dd>
<dt>Date</dt> <dd>2015-09-01 10:37:16 -0700 (Tue, 01 Sep 2015)</dd>
</dl>
<h3>Log Message</h3>
<pre>Web Inspector: add TestSuite hooks for per-testcase setup and teardown actions
https://bugs.webkit.org/show_bug.cgi?id=148641
Reviewed by Timothy Hatcher.
Source/WebInspectorUI:
It should be possible to run setup and teardown actions for each test case.
The use case here is to add listeners, set up testcase-specific state, etc.
To match how test functions work in each suite, the sync and async implementations
take functions with no arguments and(resolve, reject), respectively.
* UserInterface/Test/TestSuite.js:
(TestSuite.prototype.addTestCase):
(TestSuite.messageFromThrownObject):
(TestSuite):
(AsyncTestSuite.prototype.runTestCases): Stick the extra functions in the chain.
(AsyncTestSuite):
(SyncTestSuite.prototype.runTestCases):
(SyncTestSuite):
LayoutTests:
Extend existing unit tests to also exercise setup and teardown actions.
* inspector/unit-tests/async-test-suite.html:
* inspector/unit-tests/async-test-suite-expected.txt:
* inspector/unit-tests/sync-test-suite.html:
* inspector/unit-tests/sync-test-suite-expected.txt:</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkLayoutTestsChangeLog">trunk/LayoutTests/ChangeLog</a></li>
<li><a href="#trunkLayoutTestsinspectorunittestsasynctestsuiteexpectedtxt">trunk/LayoutTests/inspector/unit-tests/async-test-suite-expected.txt</a></li>
<li><a href="#trunkLayoutTestsinspectorunittestsasynctestsuitehtml">trunk/LayoutTests/inspector/unit-tests/async-test-suite.html</a></li>
<li><a href="#trunkLayoutTestsinspectorunittestssynctestsuiteexpectedtxt">trunk/LayoutTests/inspector/unit-tests/sync-test-suite-expected.txt</a></li>
<li><a href="#trunkLayoutTestsinspectorunittestssynctestsuitehtml">trunk/LayoutTests/inspector/unit-tests/sync-test-suite.html</a></li>
<li><a href="#trunkSourceWebInspectorUIChangeLog">trunk/Source/WebInspectorUI/ChangeLog</a></li>
<li><a href="#trunkSourceWebInspectorUIUserInterfaceTestTestSuitejs">trunk/Source/WebInspectorUI/UserInterface/Test/TestSuite.js</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkLayoutTestsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/LayoutTests/ChangeLog (189212 => 189213)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/ChangeLog 2015-09-01 17:21:21 UTC (rev 189212)
+++ trunk/LayoutTests/ChangeLog 2015-09-01 17:37:16 UTC (rev 189213)
</span><span class="lines">@@ -1,3 +1,17 @@
</span><ins>+2015-09-01 Brian Burg <bburg@apple.com>
+
+ Web Inspector: add TestSuite hooks for per-testcase setup and teardown actions
+ https://bugs.webkit.org/show_bug.cgi?id=148641
+
+ Reviewed by Timothy Hatcher.
+
+ Extend existing unit tests to also exercise setup and teardown actions.
+
+ * inspector/unit-tests/async-test-suite.html:
+ * inspector/unit-tests/async-test-suite-expected.txt:
+ * inspector/unit-tests/sync-test-suite.html:
+ * inspector/unit-tests/sync-test-suite-expected.txt:
+
</ins><span class="cx"> 2015-09-01 Zalan Bujtas <zalan@apple.com>
</span><span class="cx">
</span><span class="cx"> Repaint cleanup:
</span></span></pre></div>
<a id="trunkLayoutTestsinspectorunittestsasynctestsuiteexpectedtxt"></a>
<div class="modfile"><h4>Modified: trunk/LayoutTests/inspector/unit-tests/async-test-suite-expected.txt (189212 => 189213)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/inspector/unit-tests/async-test-suite-expected.txt 2015-09-01 17:21:21 UTC (rev 189212)
+++ trunk/LayoutTests/inspector/unit-tests/async-test-suite-expected.txt 2015-09-01 17:37:16 UTC (rev 189213)
</span><span class="lines">@@ -7,6 +7,12 @@
</span><span class="cx"> PASS: test case should require string name.
</span><span class="cx"> PASS: test case should require non-whitespace name.
</span><span class="cx"> PASS: test case should require test function.
</span><ins>+PASS: should not be able to specify non-Function `setup` parameter.
+PASS: should not be able to specify non-Function `setup` parameter.
+PASS: should not be able to specify non-Function `setup` parameter.
+PASS: should not be able to specify non-Function `teardown` parameter.
+PASS: should not be able to specify non-Function `teardown` parameter.
+PASS: should not be able to specify non-Function `teardown` parameter.
</ins><span class="cx"> PASS: should not be able to run empty test suite.
</span><span class="cx">
</span><span class="cx"> == Running test suite: AsyncTestSuite.RunTwiceSuite
</span><span class="lines">@@ -38,3 +44,36 @@
</span><span class="cx"> PASS: abortOnFailureSuite should have failed one test.
</span><span class="cx"> PASS: abortOnFailureSuite should have skipped one test.
</span><span class="cx">
</span><ins>+== Running test suite: AsyncTestSuite.SetupAndTeardown
+-- Running test setup.
+-- Running test case: TestWithSetupAndTeardown
+PASS: Test should see side effects of running setup() action.
+-- Running test teardown.
+PASS: Teardown should see side effects of running setup() action.
+
+-- Running test case: TestRunningAfterTeardown
+PASS: Test should see side effects of previous test's teardown() action.
+PASS: Promise from setupAndTeardownTestSuite.runTestCases() should resolve.
+
+== Running test suite: AsyncTestSuite.SetupException
+-- Running test setup.
+!! EXCEPTION:
+PASS: Promise from setupExceptionTestSuite.runTestCases() should reject.
+
+== Running test suite: AsyncTestSuite.SetupFailure
+-- Running test setup.
+!! EXCEPTION: undefined
+PASS: Promise from setupFailureTestSuite.runTestCases() should reject.
+
+== Running test suite: SyncTestSuite.TeardownException
+-- Running test case: TestWithExceptionDuringTeardown
+-- Running test teardown.
+!! EXCEPTION:
+PASS: Promise from teardownExceptionTestSuite.runTestCases() should reject.
+
+== Running test suite: SyncTestSuite.TeardownFailure
+-- Running test case: TestWithExceptionDuringTeardown
+-- Running test teardown.
+!! EXCEPTION: undefined
+PASS: Promise from teardownFailureTestSuite.runTestCases() should reject.
+
</ins></span></pre></div>
<a id="trunkLayoutTestsinspectorunittestsasynctestsuitehtml"></a>
<div class="modfile"><h4>Modified: trunk/LayoutTests/inspector/unit-tests/async-test-suite.html (189212 => 189213)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/inspector/unit-tests/async-test-suite.html 2015-09-01 17:21:21 UTC (rev 189212)
+++ trunk/LayoutTests/inspector/unit-tests/async-test-suite.html 2015-09-01 17:37:16 UTC (rev 189213)
</span><span class="lines">@@ -73,6 +73,66 @@
</span><span class="cx"> } catch (e) {
</span><span class="cx"> ProtocolTest.log("PASS: test case should require test function.");
</span><span class="cx"> }
</span><ins>+ try {
+ badArgsSuite.addTestCase({
+ name: "foo",
+ test: () => {},
+ setup: "astd"
+ });
+ ProtocolTest.log("FAIL: should not be able to specify non-Function `setup` parameter.");
+ } catch (e) {
+ ProtocolTest.log("PASS: should not be able to specify non-Function `setup` parameter.");
+ }
+ try {
+ badArgsSuite.addTestCase({
+ name: "foo",
+ test: () => {},
+ setup: 123
+ });
+ ProtocolTest.log("FAIL: should not be able to specify non-Function `setup` parameter.");
+ } catch (e) {
+ ProtocolTest.log("PASS: should not be able to specify non-Function `setup` parameter.");
+ }
+ try {
+ badArgsSuite.addTestCase({
+ name: "foo",
+ test: () => {},
+ setup: {}
+ });
+ ProtocolTest.log("FAIL: should not be able to specify non-Function `setup` parameter.");
+ } catch (e) {
+ ProtocolTest.log("PASS: should not be able to specify non-Function `setup` parameter.");
+ }
+ try {
+ badArgsSuite.addTestCase({
+ name: "foo",
+ test: () => {},
+ teardown: "astd"
+ });
+ ProtocolTest.log("FAIL: should not be able to specify non-Function `teardown` parameter.");
+ } catch (e) {
+ ProtocolTest.log("PASS: should not be able to specify non-Function `teardown` parameter.");
+ }
+ try {
+ badArgsSuite.addTestCase({
+ name: "foo",
+ test: () => {},
+ teardown: 123
+ });
+ ProtocolTest.log("FAIL: should not be able to specify non-Function `teardown` parameter.");
+ } catch (e) {
+ ProtocolTest.log("PASS: should not be able to specify non-Function `teardown` parameter.");
+ }
+ try {
+ badArgsSuite.addTestCase({
+ name: "foo",
+ test: () => {},
+ teardown: {}
+ });
+ ProtocolTest.log("FAIL: should not be able to specify non-Function `teardown` parameter.");
+ } catch (e) {
+ ProtocolTest.log("PASS: should not be able to specify non-Function `teardown` parameter.");
+ }
</ins><span class="cx">
</span><span class="cx"> let runEmptySuite = ProtocolTest.createAsyncSuite("AsyncTestSuite.RunEmptySuite");
</span><span class="cx"> try {
</span><span class="lines">@@ -176,6 +236,157 @@
</span><span class="cx"> return Promise.resolve(); // Continue this test.
</span><span class="cx"> });
</span><span class="cx">
</span><ins>+ var setupAndTeardownSymbol = Symbol("async-suite-setup-and-teardown-token");
+ window[setupAndTeardownSymbol] = 0;
+
+ let setupAndTeardownTestSuite = ProtocolTest.createAsyncSuite("AsyncTestSuite.SetupAndTeardown");
+ setupAndTeardownTestSuite.addTestCase({
+ name: "TestWithSetupAndTeardown",
+ description: "Check execution order for setup and teardown actions.",
+ setup: (resolve, reject) => {
+ window[setupAndTeardownSymbol] = 1;
+ resolve();
+ },
+ test: (resolve, reject) => {
+ ProtocolTest.expectThat(window[setupAndTeardownSymbol] === 1, "Test should see side effects of running setup() action.");
+ window[setupAndTeardownSymbol] = 2;
+ resolve();
+ },
+ teardown: (resolve, reject) => {
+ ProtocolTest.expectThat(window[setupAndTeardownSymbol] === 2, "Teardown should see side effects of running setup() action.");
+ window[setupAndTeardownSymbol] = 3;
+ resolve();
+ }
+ });
+ setupAndTeardownTestSuite.addTestCase({
+ name: "TestRunningAfterTeardown",
+ description: "Check execution order for test after a teardown action.",
+ test: (resolve, reject) => {
+ ProtocolTest.expectThat(window[setupAndTeardownSymbol] === 3, "Test should see side effects of previous test's teardown() action.");
+ resolve();
+ },
+ });
+
+ result = result.then(() => {
+ return setupAndTeardownTestSuite.runTestCases();
+ }).then(function resolved() {
+ ProtocolTest.log("PASS: Promise from setupAndTeardownTestSuite.runTestCases() should resolve.");
+ return Promise.resolve(); // Continue this test.
+ }, function rejected(e) {
+ ProtocolTest.log("FAIL: Promise from setupAndTeardownTestSuite.runTestCases() should resolve.");
+ return Promise.resolve(); // Continue this test.
+ });
+
+ let setupExceptionTestSuite = ProtocolTest.createAsyncSuite("AsyncTestSuite.SetupException");
+ setupExceptionTestSuite.addTestCase({
+ name: "TestWithExceptionDuringSetup",
+ description: "Check execution order for setup action that throws an exception.",
+ setup: (resolve, reject) => { throw new Error() },
+ test: (resolve, reject) => {
+ ProtocolTest.assert(false, "Test should not execute if its setup action threw an exception.");
+ reject();
+ },
+ teardown: (resolve, reject) => {
+ ProtocolTest.assert(false, "Teardown action should not execute if its setup action threw an exception.");
+ reject();
+ }
+ });
+
+ result = result.then(() => {
+ return setupExceptionTestSuite.runTestCases();
+ }).then(function resolved() {
+ ProtocolTest.log("FAIL: Promise from setupExceptionTestSuite.runTestCases() should reject.");
+ return Promise.resolve(); // Continue this test.
+ }, function rejected(e) {
+ ProtocolTest.log("PASS: Promise from setupExceptionTestSuite.runTestCases() should reject.");
+ return Promise.resolve(); // Continue this test.
+ });
+
+ let setupFailureTestSuite = ProtocolTest.createAsyncSuite("AsyncTestSuite.SetupFailure");
+ setupFailureTestSuite.addTestCase({
+ name: "TestWithFailureDuringSetup",
+ description: "Check execution order for setup action that has failed.",
+ setup: (resolve, reject) => { reject(); },
+ test: (resolve, reject) => {
+ ProtocolTest.assert(false, "Test should not execute if its setup action failed.")
+ reject();
+ },
+ teardown: (resolve, reject) => {
+ ProtocolTest.assert(false, "Teardown action should not execute if its setup action failed.")
+ reject();
+ }
+ });
+
+ result = result.then(() => {
+ return setupFailureTestSuite.runTestCases();
+ }).then(function resolved() {
+ ProtocolTest.log("FAIL: Promise from setupFailureTestSuite.runTestCases() should reject.");
+ return Promise.resolve(); // Continue this test.
+ }, function rejected(e) {
+ ProtocolTest.log("PASS: Promise from setupFailureTestSuite.runTestCases() should reject.");
+ return Promise.resolve(); // Continue this test.
+ });
+
+ let teardownExceptionTestSuite = ProtocolTest.createAsyncSuite("SyncTestSuite.TeardownException");
+ teardownExceptionTestSuite.addTestCase({
+ name: "TestWithExceptionDuringTeardown",
+ description: "Check execution order for teardown action that throws an exception.",
+ test: (resolve, reject) => { resolve(); },
+ teardown: (resolve, reject) => { throw new Error() }
+ });
+ teardownExceptionTestSuite.addTestCase({
+ name: "TestAfterTeardownException",
+ descrption: "Check execution order for test after previous test's teardown throws an exception.",
+ setup: (resolve, reject) => {
+ ProtocolTest.assert(false, "Setup action should not execute if previous test's teardown action threw an exception.");
+ reject();
+ },
+ test: (resolve, reject) => {
+ ProtocolTest.assert(false, "Test should not execute if previous test's teardown action threw an exception.");
+ reject();
+ }
+ });
+
+ result = result.then(() => {
+ return teardownExceptionTestSuite.runTestCases();
+ }).then(function resolved() {
+ ProtocolTest.log("FAIL: Promise from teardownExceptionTestSuite.runTestCases() should reject.");
+ return Promise.resolve(); // Continue this test.
+ }, function rejected(e) {
+ ProtocolTest.log("PASS: Promise from teardownExceptionTestSuite.runTestCases() should reject.");
+ return Promise.resolve(); // Continue this test.
+ });
+
+ let teardownFailureTestSuite = ProtocolTest.createAsyncSuite("SyncTestSuite.TeardownFailure");
+ teardownFailureTestSuite.addTestCase({
+ name: "TestWithExceptionDuringTeardown",
+ description: "Check execution order for teardown action that has failed.",
+ test: (resolve, reject) => { resolve(); },
+ teardown: (resolve, reject) => { reject(); },
+ });
+ teardownFailureTestSuite.addTestCase({
+ name: "TestAfterTeardownException",
+ description: "Check execution order for test after previous test's teardown throws an exception",
+ setup: (resolve, reject) => {
+ ProtocolTest.assert(false, "Setup action should not execute if previous test's teardown action failed.");
+ reject();
+ },
+ test: (resolve, reject) => {
+ ProtocolTest.assert(false, "Test should not execute if previous test's teardown action failed.");
+ reject();
+ }
+ });
+
+ result = result.then(() => {
+ return teardownFailureTestSuite.runTestCases();
+ }).then(function resolved() {
+ ProtocolTest.log("FAIL: Promise from teardownFailureTestSuite.runTestCases() should reject.");
+ return Promise.resolve(); // Continue this test.
+ }, function rejected(e) {
+ ProtocolTest.log("PASS: Promise from teardownFailureTestSuite.runTestCases() should reject.");
+ return Promise.resolve(); // Continue this test.
+ });
+
</ins><span class="cx"> // This will finish the test whether the chain was resolved or rejected.
</span><span class="cx"> result = result.then(() => { ProtocolTest.completeTest(); });
</span><span class="cx"> }
</span></span></pre></div>
<a id="trunkLayoutTestsinspectorunittestssynctestsuiteexpectedtxt"></a>
<div class="modfile"><h4>Modified: trunk/LayoutTests/inspector/unit-tests/sync-test-suite-expected.txt (189212 => 189213)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/inspector/unit-tests/sync-test-suite-expected.txt 2015-09-01 17:21:21 UTC (rev 189212)
+++ trunk/LayoutTests/inspector/unit-tests/sync-test-suite-expected.txt 2015-09-01 17:37:16 UTC (rev 189213)
</span><span class="lines">@@ -7,6 +7,12 @@
</span><span class="cx"> PASS: test case should require string name.
</span><span class="cx"> PASS: test case should require non-whitespace name.
</span><span class="cx"> PASS: test case should require test function.
</span><ins>+PASS: should not be able to specify non-Function `setup` parameter.
+PASS: should not be able to specify non-Function `setup` parameter.
+PASS: should not be able to specify non-Function `setup` parameter.
+PASS: should not be able to specify non-Function `teardown` parameter.
+PASS: should not be able to specify non-Function `teardown` parameter.
+PASS: should not be able to specify non-Function `teardown` parameter.
</ins><span class="cx"> PASS: should not be able to run empty test suite.
</span><span class="cx">
</span><span class="cx"> == Running test suite: SyncTestSuite.RunTwiceSuite
</span><span class="lines">@@ -35,3 +41,31 @@
</span><span class="cx"> PASS: abortOnFailureSuite should have failed one test.
</span><span class="cx"> PASS: abortOnFailureSuite should have skipped one test.
</span><span class="cx">
</span><ins>+== Running test suite: SyncTestSuite.SetupAndTeardown
+-- Running test setup.
+-- Running test case: TestWithSetupAndTeardown
+PASS: Test should see side effects of running setup() action.
+-- Running test teardown.
+PASS: Teardown should see side effects of running setup() action.
+
+-- Running test case: TestRunningAfterTeardown
+PASS: Test should see side effects of previous test's teardown() action.
+
+== Running test suite: SyncTestSuite.SetupException
+-- Running test setup.
+!! EXCEPTION:
+
+== Running test suite: SyncTestSuite.SetupFailure
+-- Running test setup.
+!! EXCEPTION
+
+== Running test suite: SyncTestSuite.TeardownException
+-- Running test case: TestWithExceptionDuringTeardown
+-- Running test teardown.
+!! EXCEPTION:
+
+== Running test suite: SyncTestSuite.TeardownFailure
+-- Running test case: TestWithExceptionDuringTeardown
+-- Running test teardown.
+!! EXCEPTION:
+
</ins></span></pre></div>
<a id="trunkLayoutTestsinspectorunittestssynctestsuitehtml"></a>
<div class="modfile"><h4>Modified: trunk/LayoutTests/inspector/unit-tests/sync-test-suite.html (189212 => 189213)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/inspector/unit-tests/sync-test-suite.html 2015-09-01 17:21:21 UTC (rev 189212)
+++ trunk/LayoutTests/inspector/unit-tests/sync-test-suite.html 2015-09-01 17:37:16 UTC (rev 189213)
</span><span class="lines">@@ -73,7 +73,66 @@
</span><span class="cx"> } catch (e) {
</span><span class="cx"> ProtocolTest.log("PASS: test case should require test function.");
</span><span class="cx"> }
</span><del>-
</del><ins>+ try {
+ badArgsSuite.addTestCase({
+ name: "foo",
+ test: () => {},
+ setup: "astd"
+ });
+ ProtocolTest.log("FAIL: should not be able to specify non-Function `setup` parameter.");
+ } catch (e) {
+ ProtocolTest.log("PASS: should not be able to specify non-Function `setup` parameter.");
+ }
+ try {
+ badArgsSuite.addTestCase({
+ name: "foo",
+ test: () => {},
+ setup: 123
+ });
+ ProtocolTest.log("FAIL: should not be able to specify non-Function `setup` parameter.");
+ } catch (e) {
+ ProtocolTest.log("PASS: should not be able to specify non-Function `setup` parameter.");
+ }
+ try {
+ badArgsSuite.addTestCase({
+ name: "foo",
+ test: () => {},
+ setup: Promise.resolve()
+ });
+ ProtocolTest.log("FAIL: should not be able to specify non-Function `setup` parameter.");
+ } catch (e) {
+ ProtocolTest.log("PASS: should not be able to specify non-Function `setup` parameter.");
+ }
+ try {
+ badArgsSuite.addTestCase({
+ name: "foo",
+ test: () => {},
+ teardown: "astd"
+ });
+ ProtocolTest.log("FAIL: should not be able to specify non-Function `teardown` parameter.");
+ } catch (e) {
+ ProtocolTest.log("PASS: should not be able to specify non-Function `teardown` parameter.");
+ }
+ try {
+ badArgsSuite.addTestCase({
+ name: "foo",
+ test: () => {},
+ teardown: 123
+ });
+ ProtocolTest.log("FAIL: should not be able to specify non-Function `teardown` parameter.");
+ } catch (e) {
+ ProtocolTest.log("PASS: should not be able to specify non-Function `teardown` parameter.");
+ }
+ try {
+ badArgsSuite.addTestCase({
+ name: "foo",
+ test: () => {},
+ teardown: Promise.resolve()
+ });
+ ProtocolTest.log("FAIL: should not be able to specify non-Function `teardown` parameter.");
+ } catch (e) {
+ ProtocolTest.log("PASS: should not be able to specify non-Function `teardown` parameter.");
+ }
</ins><span class="cx"> let runEmptySuite = ProtocolTest.createSyncSuite("SyncTestSuite.RunEmptySuite");
</span><span class="cx"> try {
</span><span class="cx"> runEmptySuite.runTestCases();
</span><span class="lines">@@ -154,6 +213,113 @@
</span><span class="cx"> ProtocolTest.expectThat(abortOnFailureSuite.failCount === 1, "abortOnFailureSuite should have failed one test.");
</span><span class="cx"> ProtocolTest.expectThat(abortOnFailureSuite.skipCount === 1, "abortOnFailureSuite should have skipped one test.");
</span><span class="cx">
</span><ins>+ let setupAndTeardownSymbol = Symbol("sync-suite-setup-and-teardown-token");
+ window[setupAndTeardownSymbol] = 0;
+
+ let setupAndTeardownTestSuite = ProtocolTest.createSyncSuite("SyncTestSuite.SetupAndTeardown");
+ setupAndTeardownTestSuite.addTestCase({
+ name: "TestWithSetupAndTeardown",
+ description: "Check execution order for setup and teardown actions.",
+ setup: () => {
+ window[setupAndTeardownSymbol] = 1;
+ return true;
+ },
+ test: () => {
+ ProtocolTest.expectThat(window[setupAndTeardownSymbol] === 1, "Test should see side effects of running setup() action.");
+ window[setupAndTeardownSymbol] = 2;
+ return true;
+ },
+ teardown: () => {
+ ProtocolTest.expectThat(window[setupAndTeardownSymbol] === 2, "Teardown should see side effects of running setup() action.");
+ window[setupAndTeardownSymbol] = 3;
+ return true;
+ }
+ });
+ setupAndTeardownTestSuite.addTestCase({
+ name: "TestRunningAfterTeardown",
+ description: "Check execution order for test after a teardown action.",
+ test: () => {
+ ProtocolTest.expectThat(window[setupAndTeardownSymbol] === 3, "Test should see side effects of previous test's teardown() action.");
+ return true;
+ },
+ });
+
+ setupAndTeardownTestSuite.runTestCases();
+
+ let setupExceptionTestSuite = ProtocolTest.createSyncSuite("SyncTestSuite.SetupException");
+ setupExceptionTestSuite.addTestCase({
+ name: "TestWithExceptionDuringSetup",
+ description: "Check execution order for setup action that throws an exception.",
+ setup: () => { throw new Error() },
+ test: () => {
+ ProtocolTest.assert(false, "Test should not execute if its setup action threw an exception.");
+ return false;
+ },
+ teardown: () => {
+ ProtocolTest.assert(false, "Teardown action should not execute if its setup action threw an exception.");
+ return false;
+ }
+ });
+ setupExceptionTestSuite.runTestCases();
+
+ let setupFailureTestSuite = ProtocolTest.createSyncSuite("SyncTestSuite.SetupFailure");
+ setupFailureTestSuite.addTestCase({
+ name: "TestWithFailureDuringSetup",
+ description: "Check execution order for setup action that has failed.",
+ setup: () => false,
+ test: () => {
+ ProtocolTest.assert(false, "Test should not execute if its setup action returned false.")
+ return false;
+ },
+ teardown: () => {
+ ProtocolTest.assert(false, "Teardown action should not execute if its setup action returned false.")
+ return false;
+ }
+ });
+ setupFailureTestSuite.runTestCases();
+
+ let teardownExceptionTestSuite = ProtocolTest.createSyncSuite("SyncTestSuite.TeardownException");
+ teardownExceptionTestSuite.addTestCase({
+ name: "TestWithExceptionDuringTeardown",
+ description: "Check execution order for teardown action that throws an exception.",
+ test: () => true,
+ teardown: () => { throw new Error() }
+ });
+ teardownExceptionTestSuite.addTestCase({
+ name: "TestAfterTeardownException",
+ descrption: "Check execution order for test after previous test's teardown throws an exception",
+ setup: () => {
+ ProtocolTest.assert(false, "Setup action should not execute if previous test's teardown action threw an exception.");
+ return false;
+ },
+ test: () => {
+ ProtocolTest.assert(false, "Test should not execute if previous test's teardown action threw an exception.");
+ return false;
+ }
+ });
+ teardownExceptionTestSuite.runTestCases();
+
+ let teardownFailureTestSuite = ProtocolTest.createSyncSuite("SyncTestSuite.TeardownFailure");
+ teardownFailureTestSuite.addTestCase({
+ name: "TestWithExceptionDuringTeardown",
+ description: "Check execution order for teardown action that has failed.",
+ test: () => true,
+ teardown: () => false,
+ });
+ teardownFailureTestSuite.addTestCase({
+ name: "TestAfterTeardownException",
+ descrption: "Check execution order for test after previous test's teardown throws an exception",
+ setup: () => {
+ ProtocolTest.assert(false, "Setup action should not execute if previous test's teardown action failed.");
+ return false;
+ },
+ test: () => {
+ ProtocolTest.assert(false, "Test should not execute if previous test's teardown action failed.");
+ return false;
+ }
+ });
+ teardownFailureTestSuite.runTestCases();
+
</ins><span class="cx"> ProtocolTest.completeTest();
</span><span class="cx"> }
</span><span class="cx"> </script>
</span></span></pre></div>
<a id="trunkSourceWebInspectorUIChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebInspectorUI/ChangeLog (189212 => 189213)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebInspectorUI/ChangeLog 2015-09-01 17:21:21 UTC (rev 189212)
+++ trunk/Source/WebInspectorUI/ChangeLog 2015-09-01 17:37:16 UTC (rev 189213)
</span><span class="lines">@@ -1,3 +1,24 @@
</span><ins>+2015-09-01 Brian Burg <bburg@apple.com>
+
+ Web Inspector: add TestSuite hooks for per-testcase setup and teardown actions
+ https://bugs.webkit.org/show_bug.cgi?id=148641
+
+ Reviewed by Timothy Hatcher.
+
+ It should be possible to run setup and teardown actions for each test case.
+ The use case here is to add listeners, set up testcase-specific state, etc.
+ To match how test functions work in each suite, the sync and async implementations
+ take functions with no arguments and(resolve, reject), respectively.
+
+ * UserInterface/Test/TestSuite.js:
+ (TestSuite.prototype.addTestCase):
+ (TestSuite.messageFromThrownObject):
+ (TestSuite):
+ (AsyncTestSuite.prototype.runTestCases): Stick the extra functions in the chain.
+ (AsyncTestSuite):
+ (SyncTestSuite.prototype.runTestCases):
+ (SyncTestSuite):
+
</ins><span class="cx"> 2015-09-01 Matt Baker <mattbaker@apple.com>
</span><span class="cx">
</span><span class="cx"> Web Inspector: Visual style editor shouldn't allow alpha characters in numeric input fields
</span></span></pre></div>
<a id="trunkSourceWebInspectorUIUserInterfaceTestTestSuitejs"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebInspectorUI/UserInterface/Test/TestSuite.js (189212 => 189213)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebInspectorUI/UserInterface/Test/TestSuite.js 2015-09-01 17:21:21 UTC (rev 189212)
+++ trunk/Source/WebInspectorUI/UserInterface/Test/TestSuite.js 2015-09-01 17:37:16 UTC (rev 189213)
</span><span class="lines">@@ -78,8 +78,26 @@
</span><span class="cx"> if (typeof testcase.test !== "function")
</span><span class="cx"> throw new Error("Tried to add test case without `test` function.");
</span><span class="cx">
</span><ins>+ if (testcase.setup && typeof testcase.setup !== "function")
+ throw new Error("Tried to add test case with invalid `setup` parameter (must be a function).")
+
+ if (testcase.teardown && typeof testcase.teardown !== "function")
+ throw new Error("Tried to add test case with invalid `teardown` parameter (must be a function).")
+
</ins><span class="cx"> this.testcases.push(testcase);
</span><span class="cx"> }
</span><ins>+
+ static messageFromThrownObject(e)
+ {
+ let message = e;
+ if (e instanceof Error)
+ message = e.message;
+
+ if (typeof message !== "string")
+ message = JSON.stringify(message);
+
+ return message;
+ }
</ins><span class="cx"> };
</span><span class="cx">
</span><span class="cx"> AsyncTestSuite = class AsyncTestSuite extends TestSuite
</span><span class="lines">@@ -103,32 +121,42 @@
</span><span class="cx"> this._startedRunning = true;
</span><span class="cx">
</span><span class="cx"> this._harness.log("");
</span><del>- this._harness.log("== Running test suite: " + this.name);
</del><ins>+ this._harness.log(`== Running test suite: ${this.name}`);
</ins><span class="cx">
</span><span class="cx"> // Avoid adding newlines if nothing was logged.
</span><span class="cx"> let priorLogCount = this._harness.logCount;
</span><span class="cx"> let result = this.testcases.reduce((chain, testcase, i) => {
</span><del>- return chain.then(() => {
</del><ins>+ if (testcase.setup) {
+ chain = chain.then(() => {
+ this._harness.log("-- Running test setup.");
+ return new Promise(testcase.setup);
+ });
+ }
+
+ chain = chain.then(() => {
</ins><span class="cx"> if (i > 0 && priorLogCount + 1 < this._harness.logCount)
</span><span class="cx"> this._harness.log("");
</span><span class="cx">
</span><span class="cx"> priorLogCount = this._harness.logCount;
</span><del>- this._harness.log("-- Running test case: " + testcase.name);
</del><ins>+ this._harness.log(`-- Running test case: ${testcase.name}`);
</ins><span class="cx"> this.runCount++;
</span><span class="cx"> return new Promise(testcase.test);
</span><span class="cx"> });
</span><ins>+
+ if (testcase.teardown) {
+ chain = chain.then(() => {
+ this._harness.log("-- Running test teardown.");
+ return new Promise(testcase.teardown);
+ });
+ }
+ return chain;
</ins><span class="cx"> }, Promise.resolve());
</span><span class="cx">
</span><span class="cx"> return result.catch((e) => {
</span><span class="cx"> this.failCount++;
</span><del>- let message = e;
- if (e instanceof Error)
- message = e.message;
</del><ins>+ let message = TestSuite.messageFromThrownObject(e);
+ this._harness.log(`!! EXCEPTION: ${message}`);
</ins><span class="cx">
</span><del>- if (typeof message !== "string")
- message = JSON.stringify(message);
-
- this._harness.log("!! EXCEPTION: " + message);
</del><span class="cx"> throw e; // Reject this promise by re-throwing the error.
</span><span class="cx"> });
</span><span class="cx"> }
</span><span class="lines">@@ -152,7 +180,7 @@
</span><span class="cx"> this._startedRunning = true;
</span><span class="cx">
</span><span class="cx"> this._harness.log("");
</span><del>- this._harness.log("== Running test suite: " + this.name);
</del><ins>+ this._harness.log(`== Running test suite: ${this.name}`);
</ins><span class="cx">
</span><span class="cx"> let priorLogCount = this._harness.logCount;
</span><span class="cx"> for (let i = 0; i < this.testcases.length; i++) {
</span><span class="lines">@@ -162,6 +190,22 @@
</span><span class="cx">
</span><span class="cx"> priorLogCount = this._harness.logCount;
</span><span class="cx">
</span><ins>+ // Run the setup action, if one was provided.
+ if (testcase.setup) {
+ this._harness.log("-- Running test setup.");
+ try {
+ let result = testcase.setup.call(null);
+ if (result === false) {
+ this._harness.log("!! EXCEPTION");
+ return false;
+ }
+ } catch (e) {
+ let message = TestSuite.messageFromThrownObject(e);
+ this._harness.log(`!! EXCEPTION: ${message}`);
+ return false;
+ }
+ }
+
</ins><span class="cx"> this._harness.log("-- Running test case: " + testcase.name);
</span><span class="cx"> this.runCount++;
</span><span class="cx"> try {
</span><span class="lines">@@ -172,18 +216,26 @@
</span><span class="cx"> }
</span><span class="cx"> } catch (e) {
</span><span class="cx"> this.failCount++;
</span><del>- let message = e;
- if (e instanceof Error)
- message = e.message;
- else
- e = new Error(e);
-
- if (typeof message !== "string")
- message = JSON.stringify(message);
-
- this._harness.log("!! EXCEPTION: " + message);
</del><ins>+ let message = TestSuite.messageFromThrownObject(e);
+ this._harness.log(`!! EXCEPTION: ${message}`);
</ins><span class="cx"> return false;
</span><span class="cx"> }
</span><ins>+
+ // Run the teardown action, if one was provided.
+ if (testcase.teardown) {
+ this._harness.log("-- Running test teardown.");
+ try {
+ let result = testcase.teardown.call(null);
+ if (result === false) {
+ this._harness.log("!! EXCEPTION:");
+ return false;
+ }
+ } catch (e) {
+ let message = TestSuite.messageFromThrownObject(e);
+ this._harness.log(`!! EXCEPTION: ${message}`);
+ return false;
+ }
+ }
</ins><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> return true;
</span></span></pre>
</div>
</div>
</body>
</html>