[Webkit-unassigned] [Bug 275500] New: [WebDriver] Find a way to properly send automated events into an unfocused window

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=275500

            Bug ID: 275500
           Summary: [WebDriver] Find a way to properly send automated
                    events into an unfocused window
           Product: WebKit
           Version: Safari Technology Preview
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: WebDriver
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: qianlangchen at apple.com
                CC: bburg at apple.com

Originally, our web driver relies on the browser window to be focused in order for any automated events to be successfully delivered, and it uses `[window makeKeyAndOrderToFront:]` to focus the window before each time an automated event is sent: https://github.com/WebKit/WebKit/blob/2ba7aa7e093f16512f933ba4c161edcdab869ac0/Source/WebKit/UIProcess/Automation/mac/WebAutomationSessionMac.mm#L104-L106

However, makeKeyAndOrderFront requires the NSApplication to be active, and AppKit recently stopped providing a way to force the activation of an NSApplication: https://developer.apple.com/videos/play/wwdc2023/10054?time=559

The patch https://github.com/WebKit/WebKit/pull/29683 applied a workaround for the fact that trying to focus the window may have failed before sending in an event, by letting automated mouse events bypass certain filtering checks throughout the propagation process to ensure that they get delivered to the target web page in the unfocused window.

Though, the patch did not address keyboard events, where there are certain key combos that are blocked from being sent to unfocused windows (the examples I've found are Command+Letter key combos). And also, sending events to the window and manually bypassing checks like the patch did is not ideal. We can either
   1. Find a way to force the activation of the app and focusing of the window, which may not be possible given AppKit's recent mission, or
   2. Adapt to the fact that we can no longer guarantee the window's focus before sending an event and send the events to the web view instead (similar to how there's a workaround present for automated mousemove events, but for ALL automated events).
      - ChromeDriver currently takes this approach and always send automated keyboard and mouse events to the target web views: https://chromium.googlesource.com/chromium/src/+/master/chrome/test/chromedriver/window_commands.cc#1195
      - The web driver specs did not explicit require where exactly we direct the automated events to, but it did acknowledge that there exist two options, sending them to the window versus the web view. See the second NOTE in the following section: https://w3c.github.io/webdriver/#ticks

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20240614/6f709c57/attachment.htm>