<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[198872] trunk/Source</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/198872">198872</a></dd>
<dt>Author</dt> <dd>joepeck@webkit.org</dd>
<dt>Date</dt> <dd>2016-03-30 19:03:57 -0700 (Wed, 30 Mar 2016)</dd>
</dl>
<h3>Log Message</h3>
<pre>Web Automation: Add Automation.performKeyboardInteractions
https://bugs.webkit.org/show_bug.cgi?id=155990
<rdar://problem/25426408>
Patch by Brian Burg <bburg@apple.com> on 2016-03-30
Reviewed by Timothy Hatcher.
Source/WebKit2:
Add a command that allows automation to simulate single
key strokes or insertion of an entire string, character
by character.
* UIProcess/Automation/Automation.json: Add new command.
Add a large enumeration of all virtual keys that exist
on a US 109-key keyboard layout. Add an interaction object.
* UIProcess/Automation/WebAutomationSession.cpp:
(WebKit::WebAutomationSession::performKeyboardInteractions):
Added. This method validates the incoming key interactions
from the protocol and makes a list of simulated interactions
to perform. If everything validates, then delegate the key
interaction simulations to platform-specific methods.
(WebKit::WebAutomationSession::platformSimulateKeyStroke):
(WebKit::WebAutomationSession::platformSimulateKeySequence):
Add stubs for other platforms.
* UIProcess/Automation/WebAutomationSession.h: Add new
protocol command handler and platform simulation methods.
* UIProcess/Cocoa/WebAutomationSessionCocoa.mm:
(WebKit::WebAutomationSession::platformSimulateKeyStroke):
(WebKit::WebAutomationSession::platformSimulateKeySequence):
These methods implement keyboard simulation for AppKit, used
by the Mac port. In the keystroke case, figure out the AppKit
keyCode for the key as well as any key modifiers that should
be included with the key event. Keep track of sticky modifiers
and update the session state appropriately. In the key sequence
case, split the string into combining character sequences and
send a 'key up/down'. This is a weird way to send non-ASCII
text, so a better alternative should be explored in the future.
Source/WTF:
Add a missing NSEventType declaration.
* wtf/mac/AppKitCompatibilityDeclarations.h:</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourceWTFChangeLog">trunk/Source/WTF/ChangeLog</a></li>
<li><a href="#trunkSourceWTFwtfmacAppKitCompatibilityDeclarationsh">trunk/Source/WTF/wtf/mac/AppKitCompatibilityDeclarations.h</a></li>
<li><a href="#trunkSourceWebKit2ChangeLog">trunk/Source/WebKit2/ChangeLog</a></li>
<li><a href="#trunkSourceWebKit2UIProcessAutomationAutomationjson">trunk/Source/WebKit2/UIProcess/Automation/Automation.json</a></li>
<li><a href="#trunkSourceWebKit2UIProcessAutomationWebAutomationSessioncpp">trunk/Source/WebKit2/UIProcess/Automation/WebAutomationSession.cpp</a></li>
<li><a href="#trunkSourceWebKit2UIProcessAutomationWebAutomationSessionh">trunk/Source/WebKit2/UIProcess/Automation/WebAutomationSession.h</a></li>
<li><a href="#trunkSourceWebKit2UIProcessCocoaWebAutomationSessionCocoamm">trunk/Source/WebKit2/UIProcess/Cocoa/WebAutomationSessionCocoa.mm</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourceWTFChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WTF/ChangeLog (198871 => 198872)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WTF/ChangeLog 2016-03-31 01:36:30 UTC (rev 198871)
+++ trunk/Source/WTF/ChangeLog 2016-03-31 02:03:57 UTC (rev 198872)
</span><span class="lines">@@ -1,3 +1,15 @@
</span><ins>+2016-03-30 Brian Burg <bburg@apple.com>
+
+ Web Automation: Add Automation.performKeyboardInteractions
+ https://bugs.webkit.org/show_bug.cgi?id=155990
+ <rdar://problem/25426408>
+
+ Reviewed by Timothy Hatcher.
+
+ Add a missing NSEventType declaration.
+
+ * wtf/mac/AppKitCompatibilityDeclarations.h:
+
</ins><span class="cx"> 2016-03-29 Benjamin Poulain <bpoulain@apple.com>
</span><span class="cx">
</span><span class="cx"> [WTF] Removing a smart pointer from HashTable issues two stores to the same location
</span></span></pre></div>
<a id="trunkSourceWTFwtfmacAppKitCompatibilityDeclarationsh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WTF/wtf/mac/AppKitCompatibilityDeclarations.h (198871 => 198872)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WTF/wtf/mac/AppKitCompatibilityDeclarations.h 2016-03-31 01:36:30 UTC (rev 198871)
+++ trunk/Source/WTF/wtf/mac/AppKitCompatibilityDeclarations.h 2016-03-31 02:03:57 UTC (rev 198872)
</span><span class="lines">@@ -69,6 +69,7 @@
</span><span class="cx"> static const NSEventModifierFlags NSEventModifierFlagFunction = NSFunctionKeyMask;
</span><span class="cx"> static const NSEventModifierFlags NSEventModifierFlagNumericPad = NSNumericPadKeyMask;
</span><span class="cx"> static const NSEventModifierFlags NSEventModifierFlagShift = NSShiftKeyMask;
</span><ins>+static const NSEventModifierFlags NSEventModifierFlagHelp = NSHelpKeyMask;
</ins><span class="cx">
</span><span class="cx"> static const NSEventType NSEventTypeFlagsChanged = NSFlagsChanged;
</span><span class="cx"> static const NSEventType NSEventTypeKeyDown = NSKeyDown;
</span></span></pre></div>
<a id="trunkSourceWebKit2ChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit2/ChangeLog (198871 => 198872)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit2/ChangeLog 2016-03-31 01:36:30 UTC (rev 198871)
+++ trunk/Source/WebKit2/ChangeLog 2016-03-31 02:03:57 UTC (rev 198872)
</span><span class="lines">@@ -1,3 +1,45 @@
</span><ins>+2016-03-30 Brian Burg <bburg@apple.com>
+
+ Web Automation: Add Automation.performKeyboardInteractions
+ https://bugs.webkit.org/show_bug.cgi?id=155990
+ <rdar://problem/25426408>
+
+ Reviewed by Timothy Hatcher.
+
+ Add a command that allows automation to simulate single
+ key strokes or insertion of an entire string, character
+ by character.
+
+ * UIProcess/Automation/Automation.json: Add new command.
+ Add a large enumeration of all virtual keys that exist
+ on a US 109-key keyboard layout. Add an interaction object.
+
+ * UIProcess/Automation/WebAutomationSession.cpp:
+ (WebKit::WebAutomationSession::performKeyboardInteractions):
+ Added. This method validates the incoming key interactions
+ from the protocol and makes a list of simulated interactions
+ to perform. If everything validates, then delegate the key
+ interaction simulations to platform-specific methods.
+
+ (WebKit::WebAutomationSession::platformSimulateKeyStroke):
+ (WebKit::WebAutomationSession::platformSimulateKeySequence):
+ Add stubs for other platforms.
+
+ * UIProcess/Automation/WebAutomationSession.h: Add new
+ protocol command handler and platform simulation methods.
+
+ * UIProcess/Cocoa/WebAutomationSessionCocoa.mm:
+ (WebKit::WebAutomationSession::platformSimulateKeyStroke):
+ (WebKit::WebAutomationSession::platformSimulateKeySequence):
+ These methods implement keyboard simulation for AppKit, used
+ by the Mac port. In the keystroke case, figure out the AppKit
+ keyCode for the key as well as any key modifiers that should
+ be included with the key event. Keep track of sticky modifiers
+ and update the session state appropriately. In the key sequence
+ case, split the string into combining character sequences and
+ send a 'key up/down'. This is a weird way to send non-ASCII
+ text, so a better alternative should be explored in the future.
+
</ins><span class="cx"> 2016-03-30 Brady Eidson <beidson@apple.com>
</span><span class="cx">
</span><span class="cx"> Make BlobData use ThreadSafeSharedBuffer instead of RawData.
</span></span></pre></div>
<a id="trunkSourceWebKit2UIProcessAutomationAutomationjson"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit2/UIProcess/Automation/Automation.json (198871 => 198872)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit2/UIProcess/Automation/Automation.json 2016-03-31 01:36:30 UTC (rev 198871)
+++ trunk/Source/WebKit2/UIProcess/Automation/Automation.json 2016-03-31 02:03:57 UTC (rev 198872)
</span><span class="lines">@@ -103,6 +103,88 @@
</span><span class="cx"> "Meta",
</span><span class="cx"> "Alt"
</span><span class="cx"> ]
</span><ins>+ },
+ {
+ "id": "VirtualKey",
+ "type": "string",
+ "description": "Enumerates different platform-independent virtual keys on a physical keyboard whose input via keyboard may or may not produce characters.",
+ "enum": [
+ "Shift",
+ "Control",
+ "Alternate",
+ "Meta",
+ "Command",
+ "Cancel",
+ "Help",
+ "Backspace",
+ "Tab",
+ "Clear",
+ "Enter",
+ "Pause",
+ "Escape",
+ "PageUp",
+ "PageDown",
+ "End",
+ "Home",
+ "LeftArrow",
+ "UpArrow",
+ "RightArrow",
+ "DownArrow",
+ "Insert",
+ "Delete",
+ "Space",
+ "Semicolon",
+ "Equals",
+ "Return",
+ "NumberPad0",
+ "NumberPad1",
+ "NumberPad2",
+ "NumberPad3",
+ "NumberPad4",
+ "NumberPad5",
+ "NumberPad6",
+ "NumberPad7",
+ "NumberPad8",
+ "NumberPad9",
+ "NumberPadMultiply",
+ "NumberPadAdd",
+ "NumberPadSeparator",
+ "NumberPadSubtract",
+ "NumberPadDecimal",
+ "NumberPadDivide",
+ "Function1",
+ "Function2",
+ "Function3",
+ "Function4",
+ "Function5",
+ "Function6",
+ "Function7",
+ "Function8",
+ "Function9",
+ "Function10",
+ "Function11",
+ "Function12"
+ ]
+ },
+ {
+ "id": "KeyboardInteractionType",
+ "type": "string",
+ "description": "Enumerates different ways of interacting with a keyboard device. 'InsertByKey' implies that a separate KeyDown and KeyUp event are produced for each combining character sequence, regardless of the actual keystrokes required to produce the character sequence.",
+ "enum": [
+ "KeyPress",
+ "KeyRelease",
+ "InsertByKey"
+ ]
+ },
+ {
+ "id": "KeyboardInteraction",
+ "type": "object",
+ "description": "A single step in a key sequence. A step can contain a key up/down of a virtual key, or a sequence of Unicode code points that are delivered to the page at grapheme cluster boundaries. Either a 'key' or 'text' property must be specified.",
+ "properties": [
+ { "name": "type", "$ref": "KeyboardInteractionType", "description": "The type of interaction to be performed by a step." },
+ { "name": "key", "$ref": "VirtualKey", "optional": true, "description": "A virtual key to be used to perform the specified interaction." },
+ { "name": "text", "type": "string", "optional": true, "description": "A unicode string to be delivered to the page. The sequence of key events is determined by splitting the string at grapheme cluster boundaries." }
+ ]
</ins><span class="cx"> }
</span><span class="cx"> ],
</span><span class="cx"> "commands": [
</span><span class="lines">@@ -220,6 +302,14 @@
</span><span class="cx"> ]
</span><span class="cx"> },
</span><span class="cx"> {
</span><ins>+ "name": "performKeyboardInteractions",
+ "description": "Simulates delivering the results of pressing one or more keyboard keys together or successively.",
+ "parameters": [
+ { "name": "handle", "$ref": "BrowsingContextHandle", "description": "The handle for the browsing context which should recieve key." },
+ { "name": "interactions", "type": "array", "items": { "$ref": "KeyboardInteraction" }, "description": "An ordered list of key sequences to be delivered using native key events." }
+ ]
+ },
+ {
</ins><span class="cx"> "name": "resolveChildFrameHandle",
</span><span class="cx"> "description": "Determines the <code>FrameHandle</code> based on the ordinal, name or node handle of a child frame.",
</span><span class="cx"> "parameters": [
</span></span></pre></div>
<a id="trunkSourceWebKit2UIProcessAutomationWebAutomationSessioncpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit2/UIProcess/Automation/WebAutomationSession.cpp (198871 => 198872)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit2/UIProcess/Automation/WebAutomationSession.cpp 2016-03-31 01:36:30 UTC (rev 198871)
+++ trunk/Source/WebKit2/UIProcess/Automation/WebAutomationSession.cpp 2016-03-31 02:03:57 UTC (rev 198872)
</span><span class="lines">@@ -708,10 +708,85 @@
</span><span class="cx"> #endif // USE(APPKIT)
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void WebAutomationSession::performKeyboardInteractions(ErrorString& errorString, const String& handle, const Inspector::InspectorArray& interactions)
+{
</ins><span class="cx"> #if !USE(APPKIT)
</span><ins>+ FAIL_WITH_PREDEFINED_ERROR_MESSAGE(NotImplemented);
+#else
+ WebPageProxy* page = webPageProxyForHandle(handle);
+ if (!page)
+ FAIL_WITH_PREDEFINED_ERROR_MESSAGE(WindowNotFound);
+
+ if (!interactions.length())
+ FAIL_WITH_PREDEFINED_ERROR_MESSAGE(InvalidParameter);
+
+ // Validate all of the parameters before performing any interactions with the browsing context under test.
+ Vector<std::function<void()>> actionsToPerform(interactions.length());
+
+ for (auto it = interactions.begin(); it != interactions.end(); ++it) {
+ RefPtr<InspectorObject> interactionObject;
+ if (!it->get()->asObject(interactionObject))
+ FAIL_WITH_PREDEFINED_ERROR_MESSAGE(InvalidParameter);
+
+ String interactionTypeString;
+ if (!interactionObject->getString(ASCIILiteral("type"), interactionTypeString))
+ FAIL_WITH_PREDEFINED_ERROR_MESSAGE(InvalidParameter);
+ auto interactionType = Inspector::Protocol::parseEnumValueFromString<Inspector::Protocol::Automation::KeyboardInteractionType>(interactionTypeString);
+ if (!interactionType)
+ FAIL_WITH_PREDEFINED_ERROR_MESSAGE(InvalidParameter);
+
+ String virtualKeyString;
+ bool foundVirtualKey = interactionObject->getString(ASCIILiteral("key"), virtualKeyString);
+ if (foundVirtualKey) {
+ auto virtualKey = Inspector::Protocol::parseEnumValueFromString<Inspector::Protocol::Automation::VirtualKey>(virtualKeyString);
+ if (!virtualKey)
+ FAIL_WITH_PREDEFINED_ERROR_MESSAGE(InvalidParameter);
+
+ actionsToPerform.append([this, page, interactionType, virtualKey] {
+ platformSimulateKeyStroke(*page, interactionType.value(), virtualKey.value());
+ });
+ }
+
+ String keySequence;
+ bool foundKeySequence = interactionObject->getString(ASCIILiteral("text"), keySequence);
+ if (foundKeySequence) {
+ switch (interactionType.value()) {
+ case Inspector::Protocol::Automation::KeyboardInteractionType::KeyPress:
+ case Inspector::Protocol::Automation::KeyboardInteractionType::KeyRelease:
+ // 'KeyPress' and 'KeyRelease' are meant for a virtual key and are not supported for a string (sequence of codepoints).
+ FAIL_WITH_PREDEFINED_ERROR_MESSAGE(InvalidParameter);
+
+ case Inspector::Protocol::Automation::KeyboardInteractionType::InsertByKey:
+ actionsToPerform.append([this, page, keySequence] {
+ platformSimulateKeySequence(*page, keySequence);
+ });
+ break;
+ }
+ }
+
+ if (!foundVirtualKey && !foundKeySequence)
+ FAIL_WITH_PREDEFINED_ERROR_MESSAGE(MissingParameter);
+
+ ASSERT(actionsToPerform.size());
+ for (auto& action : actionsToPerform)
+ action();
+ }
+
+#endif // USE(APPKIT)
+}
+
+#if !USE(APPKIT)
</ins><span class="cx"> void WebAutomationSession::platformSimulateMouseInteraction(WebKit::WebPageProxy&, const WebCore::IntPoint&, Inspector::Protocol::Automation::MouseInteraction, Inspector::Protocol::Automation::MouseButton, WebEvent::Modifiers)
</span><span class="cx"> {
</span><span class="cx"> }
</span><ins>+
+void WebAutomationSession::platformSimulateKeyStroke(WebPageProxy&, Inspector::Protocol::Automation::KeyboardInteractionType, Inspector::Protocol::Automation::VirtualKey)
+{
+}
+
+void WebAutomationSession::platformSimulateKeySequence(WebPageProxy&, const String&)
+{
+}
</ins><span class="cx"> #endif // !USE(APPKIT)
</span><span class="cx">
</span><span class="cx"> } // namespace WebKit
</span></span></pre></div>
<a id="trunkSourceWebKit2UIProcessAutomationWebAutomationSessionh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit2/UIProcess/Automation/WebAutomationSession.h (198871 => 198872)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit2/UIProcess/Automation/WebAutomationSession.h 2016-03-31 01:36:30 UTC (rev 198871)
+++ trunk/Source/WebKit2/UIProcess/Automation/WebAutomationSession.h 2016-03-31 02:03:57 UTC (rev 198872)
</span><span class="lines">@@ -104,6 +104,7 @@
</span><span class="cx"> void reloadBrowsingContext(Inspector::ErrorString&, const String&) override;
</span><span class="cx"> void evaluateJavaScriptFunction(Inspector::ErrorString&, const String& browsingContextHandle, const String* optionalFrameHandle, const String& function, const Inspector::InspectorArray& arguments, bool expectsImplicitCallbackArgument, Ref<Inspector::AutomationBackendDispatcherHandler::EvaluateJavaScriptFunctionCallback>&&) override;
</span><span class="cx"> void performMouseInteraction(Inspector::ErrorString&, const String& handle, const Inspector::InspectorObject& requestedPosition, const String& mouseButton, const String& mouseInteraction, const Inspector::InspectorArray& keyModifiers, RefPtr<Inspector::Protocol::Automation::Point>& updatedPosition) override;
</span><ins>+ void performKeyboardInteractions(Inspector::ErrorString&, const String& handle, const Inspector::InspectorArray& interactions) override;
</ins><span class="cx"> void resolveChildFrameHandle(Inspector::ErrorString&, const String& browsingContextHandle, const String* optionalFrameHandle, const int* optionalOrdinal, const String* optionalName, const String* optionalNodeHandle, Ref<ResolveChildFrameHandleCallback>&&) override;
</span><span class="cx"> void resolveParentFrameHandle(Inspector::ErrorString&, const String& browsingContextHandle, const String& frameHandle, Ref<ResolveParentFrameHandleCallback>&&) override;
</span><span class="cx"> void computeElementLayout(Inspector::ErrorString&, const String& browsingContextHandle, const String& frameHandle, const String& nodeHandle, const bool* optionalScrollIntoViewIfNeeded, const bool* useViewportCoordinates, Ref<Inspector::AutomationBackendDispatcherHandler::ComputeElementLayoutCallback>&&) override;
</span><span class="lines">@@ -137,6 +138,10 @@
</span><span class="cx">
</span><span class="cx"> // Platform-specific helper methods.
</span><span class="cx"> void platformSimulateMouseInteraction(WebPageProxy&, const WebCore::IntPoint& viewPosition, Inspector::Protocol::Automation::MouseInteraction, Inspector::Protocol::Automation::MouseButton, WebEvent::Modifiers);
</span><ins>+ // Simulates a single virtual key being pressed, such as Control, F-keys, Numpad keys, etc. as allowed by the protocol.
+ void platformSimulateKeyStroke(WebPageProxy&, Inspector::Protocol::Automation::KeyboardInteractionType, Inspector::Protocol::Automation::VirtualKey);
+ // Simulates key presses to produce the codepoints in a string. One or more code points are delivered atomically at grapheme cluster boundaries.
+ void platformSimulateKeySequence(WebPageProxy&, const String&);
</ins><span class="cx">
</span><span class="cx"> #if USE(APPKIT)
</span><span class="cx"> void sendSynthesizedEventsToPage(WebPageProxy&, NSArray *eventsToSend);
</span></span></pre></div>
<a id="trunkSourceWebKit2UIProcessCocoaWebAutomationSessionCocoamm"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit2/UIProcess/Cocoa/WebAutomationSessionCocoa.mm (198871 => 198872)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit2/UIProcess/Cocoa/WebAutomationSessionCocoa.mm 2016-03-31 01:36:30 UTC (rev 198871)
+++ trunk/Source/WebKit2/UIProcess/Cocoa/WebAutomationSessionCocoa.mm 2016-03-31 02:03:57 UTC (rev 198872)
</span><span class="lines">@@ -33,6 +33,10 @@
</span><span class="cx"> #import <WebCore/PlatformMouseEvent.h>
</span><span class="cx"> #import <objc/runtime.h>
</span><span class="cx">
</span><ins>+#if USE(APPKIT)
+#import <HIToolbox/Events.h>
+#endif
+
</ins><span class="cx"> using namespace WebCore;
</span><span class="cx">
</span><span class="cx"> namespace WebKit {
</span><span class="lines">@@ -133,6 +137,323 @@
</span><span class="cx"> sendSynthesizedEventsToPage(page, eventsToBeSent.get());
</span><span class="cx"> }
</span><span class="cx">
</span><ins>+void WebAutomationSession::platformSimulateKeyStroke(WebPageProxy& page, Inspector::Protocol::Automation::KeyboardInteractionType interaction, Inspector::Protocol::Automation::VirtualKey key)
+{
+ // If true, the key's modifier flags should affect other events while pressed down.
+ bool isStickyModifier = false;
+ // The modifiers changed by the virtual key when it is pressed or released.
+ // The mapping from keys to modifiers is specified in the documentation for NSEvent.
+ NSEventModifierFlags changedModifiers = 0;
+ // The likely keyCode for the virtual key as defined in <HIToolbox/Events.h>.
+ int keyCode = 0;
+ // Typical characters produced by the virtual key, if any.
+ NSString *characters;
+
+ // FIXME: this function and the Automation protocol enum should probably adopt key names
+ // from W3C UIEvents standard. For more details: https://w3c.github.io/uievents-code/
+
+ switch (key) {
+ case Inspector::Protocol::Automation::VirtualKey::Shift:
+ isStickyModifier = true;
+ changedModifiers |= NSEventModifierFlagShift;
+ keyCode = kVK_Shift;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Control:
+ isStickyModifier = true;
+ changedModifiers |= NSEventModifierFlagControl;
+ keyCode = kVK_Control;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Alternate:
+ isStickyModifier = true;
+ changedModifiers |= NSEventModifierFlagOption;
+ keyCode = kVK_Option;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Meta:
+ // The 'meta' key does not exist on Apple keyboards and is usually
+ // mapped to the Command key when using third-party keyboards.
+ case Inspector::Protocol::Automation::VirtualKey::Command:
+ isStickyModifier = true;
+ changedModifiers |= NSEventModifierFlagCommand;
+ keyCode = kVK_Command;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Help:
+ changedModifiers |= NSEventModifierFlagHelp | NSEventModifierFlagFunction;
+ keyCode = kVK_Help;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Backspace:
+ keyCode = kVK_Delete;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Tab:
+ keyCode = kVK_Tab;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Clear:
+ changedModifiers |= NSEventModifierFlagNumericPad;
+ keyCode = kVK_ANSI_KeypadClear;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Enter:
+ keyCode = kVK_ANSI_KeypadEnter;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Pause:
+ // The 'pause' key does not exist on Apple keyboards and has no keycode.
+ // The semantics are unclear so just abort and do nothing.
+ return;
+ case Inspector::Protocol::Automation::VirtualKey::Cancel:
+ // The 'cancel' key does not exist on Apple keyboards and has no keycode.
+ // According to the internet its functionality is similar to 'Escape'.
+ case Inspector::Protocol::Automation::VirtualKey::Escape:
+ keyCode = kVK_Escape;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::PageUp:
+ changedModifiers |= NSEventModifierFlagFunction;
+ keyCode = kVK_PageUp;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::PageDown:
+ changedModifiers |= NSEventModifierFlagFunction;
+ keyCode = kVK_PageDown;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::End:
+ changedModifiers |= NSEventModifierFlagFunction;
+ keyCode = kVK_End;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Home:
+ changedModifiers |= NSEventModifierFlagFunction;
+ keyCode = kVK_Home;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::LeftArrow:
+ changedModifiers |= NSEventModifierFlagNumericPad | NSEventModifierFlagFunction;
+ keyCode = kVK_LeftArrow;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::UpArrow:
+ changedModifiers |= NSEventModifierFlagNumericPad | NSEventModifierFlagFunction;
+ keyCode = kVK_UpArrow;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::RightArrow:
+ changedModifiers |= NSEventModifierFlagNumericPad | NSEventModifierFlagFunction;
+ keyCode = kVK_RightArrow;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::DownArrow:
+ changedModifiers |= NSEventModifierFlagNumericPad | NSEventModifierFlagFunction;
+ keyCode = kVK_DownArrow;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Insert:
+ // The 'insert' key does not exist on Apple keyboards and has no keycode.
+ // The semantics are unclear so just abort and do nothing.
+ return;
+ case Inspector::Protocol::Automation::VirtualKey::Delete:
+ changedModifiers |= NSEventModifierFlagFunction;
+ keyCode = kVK_ForwardDelete;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Space:
+ keyCode = kVK_Space;
+ characters = @" ";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Semicolon:
+ keyCode = kVK_ANSI_Semicolon;
+ characters = @";";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Equals:
+ keyCode = kVK_ANSI_Equal;
+ characters = @"=";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Return:
+ keyCode = kVK_Return;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::NumberPad0:
+ changedModifiers |= NSEventModifierFlagNumericPad;
+ keyCode = kVK_ANSI_Keypad0;
+ characters = @"0";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::NumberPad1:
+ changedModifiers |= NSEventModifierFlagNumericPad;
+ keyCode = kVK_ANSI_Keypad1;
+ characters = @"1";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::NumberPad2:
+ changedModifiers |= NSEventModifierFlagNumericPad;
+ keyCode = kVK_ANSI_Keypad2;
+ characters = @"2";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::NumberPad3:
+ changedModifiers |= NSEventModifierFlagNumericPad;
+ keyCode = kVK_ANSI_Keypad3;
+ characters = @"3";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::NumberPad4:
+ changedModifiers |= NSEventModifierFlagNumericPad;
+ keyCode = kVK_ANSI_Keypad4;
+ characters = @"4";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::NumberPad5:
+ changedModifiers |= NSEventModifierFlagNumericPad;
+ keyCode = kVK_ANSI_Keypad5;
+ characters = @"5";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::NumberPad6:
+ changedModifiers |= NSEventModifierFlagNumericPad;
+ keyCode = kVK_ANSI_Keypad6;
+ characters = @"6";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::NumberPad7:
+ changedModifiers |= NSEventModifierFlagNumericPad;
+ keyCode = kVK_ANSI_Keypad7;
+ characters = @"7";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::NumberPad8:
+ changedModifiers |= NSEventModifierFlagNumericPad;
+ keyCode = kVK_ANSI_Keypad8;
+ characters = @"8";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::NumberPad9:
+ changedModifiers |= NSEventModifierFlagNumericPad;
+ keyCode = kVK_ANSI_Keypad9;
+ characters = @"9";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::NumberPadMultiply:
+ changedModifiers |= NSEventModifierFlagNumericPad;
+ keyCode = kVK_ANSI_KeypadMultiply;
+ characters = @"*";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::NumberPadAdd:
+ changedModifiers |= NSEventModifierFlagNumericPad;
+ keyCode = kVK_ANSI_KeypadPlus;
+ characters = @"+";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::NumberPadSubtract:
+ changedModifiers |= NSEventModifierFlagNumericPad;
+ keyCode = kVK_ANSI_KeypadMinus;
+ characters = @"-";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::NumberPadSeparator:
+ changedModifiers |= NSEventModifierFlagNumericPad;
+ // The 'Separator' key is only present on a few international keyboards.
+ // It is usually mapped to the same character as Decimal ('.' or ',').
+ FALLTHROUGH;
+ case Inspector::Protocol::Automation::VirtualKey::NumberPadDecimal:
+ changedModifiers |= NSEventModifierFlagNumericPad;
+ keyCode = kVK_ANSI_KeypadDecimal;
+ // FIXME: this might be locale-dependent. See the above comment.
+ characters = @".";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::NumberPadDivide:
+ changedModifiers |= NSEventModifierFlagNumericPad;
+ keyCode = kVK_ANSI_KeypadDivide;
+ characters = @"/";
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Function1:
+ changedModifiers |= NSEventModifierFlagFunction;
+ keyCode = kVK_F1;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Function2:
+ changedModifiers |= NSEventModifierFlagFunction;
+ keyCode = kVK_F2;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Function3:
+ changedModifiers |= NSEventModifierFlagFunction;
+ keyCode = kVK_F3;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Function4:
+ changedModifiers |= NSEventModifierFlagFunction;
+ keyCode = kVK_F4;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Function5:
+ changedModifiers |= NSEventModifierFlagFunction;
+ keyCode = kVK_F5;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Function6:
+ changedModifiers |= NSEventModifierFlagFunction;
+ keyCode = kVK_F6;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Function7:
+ changedModifiers |= NSEventModifierFlagFunction;
+ keyCode = kVK_F7;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Function8:
+ changedModifiers |= NSEventModifierFlagFunction;
+ keyCode = kVK_F8;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Function9:
+ changedModifiers |= NSEventModifierFlagFunction;
+ keyCode = kVK_F9;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Function10:
+ changedModifiers |= NSEventModifierFlagFunction;
+ keyCode = kVK_F10;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Function11:
+ changedModifiers |= NSEventModifierFlagFunction;
+ keyCode = kVK_F11;
+ break;
+ case Inspector::Protocol::Automation::VirtualKey::Function12:
+ changedModifiers |= NSEventModifierFlagFunction;
+ keyCode = kVK_F12;
+ break;
+ }
+
+ auto eventsToBeSent = adoptNS([[NSMutableArray alloc] init]);
+
+ ASSERT(isStickyModifier || interaction == Inspector::Protocol::Automation::KeyboardInteractionType::KeyPress);
+
+ NSEventModifierFlags existingModifiers = [NSEvent modifierFlags];
+ NSEventModifierFlags updatedModifiers = 0;
+ NSTimeInterval timestamp = [NSDate timeIntervalSinceReferenceDate];
+ NSWindow *window = page.platformWindow();
+ NSInteger windowNumber = window.windowNumber;
+ NSPoint eventPosition = NSMakePoint(0, window.frame.size.height);
+
+ switch (interaction) {
+ case Inspector::Protocol::Automation::KeyboardInteractionType::KeyPress: {
+ NSEventType eventType = isStickyModifier ? NSEventTypeFlagsChanged : NSEventTypeKeyDown;
+ updatedModifiers = existingModifiers | changedModifiers;
+ [eventsToBeSent addObject:[NSEvent keyEventWithType:eventType location:eventPosition modifierFlags:updatedModifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:characters charactersIgnoringModifiers:characters isARepeat:NO keyCode:keyCode]];
+ break;
+ }
+ case Inspector::Protocol::Automation::KeyboardInteractionType::KeyRelease: {
+ NSEventType eventType = isStickyModifier ? NSEventTypeFlagsChanged : NSEventTypeKeyUp;
+ updatedModifiers = existingModifiers & ~changedModifiers;
+ [eventsToBeSent addObject:[NSEvent keyEventWithType:eventType location:eventPosition modifierFlags:updatedModifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:characters charactersIgnoringModifiers:characters isARepeat:NO keyCode:keyCode]];
+ break;
+ }
+ case Inspector::Protocol::Automation::KeyboardInteractionType::InsertByKey: {
+ // Sticky modifiers should either be 'KeyPress' or 'KeyRelease'.
+ ASSERT(!isStickyModifier);
+ if (isStickyModifier)
+ return;
+
+ updatedModifiers = existingModifiers | changedModifiers;
+ [eventsToBeSent addObject:[NSEvent keyEventWithType:NSEventTypeKeyDown location:eventPosition modifierFlags:updatedModifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:characters charactersIgnoringModifiers:characters isARepeat:NO keyCode:keyCode]];
+ [eventsToBeSent addObject:[NSEvent keyEventWithType:NSEventTypeKeyUp location:eventPosition modifierFlags:updatedModifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:characters charactersIgnoringModifiers:characters isARepeat:NO keyCode:keyCode]];
+ break;
+ }
+ }
+
+ sendSynthesizedEventsToPage(page, eventsToBeSent.get());
+}
+
+void WebAutomationSession::platformSimulateKeySequence(WebPageProxy& page, const String& keySequence)
+{
+ auto eventsToBeSent = adoptNS([[NSMutableArray alloc] init]);
+
+ // Split the text into combining character sequences and send each separately.
+ // This has no similarity to how keyboards work when inputting complex text.
+ // This command is more similar to the 'insertText:' editing command, except
+ // that this emits keyup/keydown/keypress events for roughly each character.
+ // This API should move more towards that direction in the future.
+ NSString *text = keySequence;
+
+ NSEventModifierFlags modifiers = [NSEvent modifierFlags];
+ NSTimeInterval timestamp = [NSDate timeIntervalSinceReferenceDate];
+ NSWindow *window = page.platformWindow();
+ NSInteger windowNumber = window.windowNumber;
+ NSPoint eventPosition = NSMakePoint(0, window.frame.size.height);
+
+ [text enumerateSubstringsInRange:NSMakeRange(0, text.length) options:NSStringEnumerationByComposedCharacterSequences usingBlock:^(NSString * _Nullable substring, NSRange substringRange, NSRange enclosingRange, BOOL * _Nonnull stop) {
+ [eventsToBeSent addObject:[NSEvent keyEventWithType:NSEventTypeKeyDown location:eventPosition modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:substring charactersIgnoringModifiers:substring isARepeat:NO keyCode:0]];
+ [eventsToBeSent addObject:[NSEvent keyEventWithType:NSEventTypeKeyUp location:eventPosition modifierFlags:modifiers timestamp:timestamp windowNumber:windowNumber context:nil characters:substring charactersIgnoringModifiers:substring isARepeat:NO keyCode:0]];
+ }];
+
+ sendSynthesizedEventsToPage(page, eventsToBeSent.get());
+}
+
</ins><span class="cx"> #endif // USE(APPKIT)
</span><span class="cx">
</span><span class="cx"> } // namespace WebKit
</span></span></pre>
</div>
</div>
</body>
</html>