[Webkit-unassigned] [Bug 260189] New: AX: Voice Over cannot read the state changes when users interacts with some HTML element in PWA Mac

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=260189

            Bug ID: 260189
           Summary: AX: Voice Over cannot read the state changes when
                    users interacts with some HTML element in PWA Mac
           Product: WebKit
           Version: WebKit Nightly Build
          Hardware: Mac (Intel)
                OS: macOS 13
            Status: NEW
          Severity: Major
          Priority: P2
         Component: Accessibility
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: xiaofeihan at microsoft.com
                CC: andresg_22 at apple.com,
                    webkit-bug-importer at group.apple.com

Created attachment 467274

  --> https://bugs.webkit.org/attachment.cgi?id=467274&action=review

accessibility inspector on PWA app

Recently, I found there are some problems with Voice Over on PWA app: Voice Over cannot read the state changes when users interacts with some HTML element.
I am not sure this bug is a chromium bug or Voice Over bug. I have reported the bug(https://bugs.chromium.org/p/chromium/issues/detail?id=1470130&q=pwa%20voiceover&can=2) to chromium, where I wrote the reproduce steps and video. 

I am working on the bug, but I don’t have much of an idea. So report here to see if you have some insight on this issue.

# How to reproduce the bug
1. Install PWA-test (pwa-a11-test.netlify.app) PWA app
2. Open the PWA app
3. Turn on the VoiceOver
4. Pressing `Space` on the checkbox element
(Note: I only use checkbox element as a example, other HTML element also have the similar problem, such as Pressing Left/Right on slider element)
Expected: Voice Over read the value changes, just like the behavior shown in the browser.
Actual: Voice Over do nothing.

# Chromium code analysis
The bug only exist in PWA scenario, it’s fine for browser scenario.

FYI, PWA Mac app has its own process(called app shim process), which spawn browser process(think it as Chrome app) and  then communicate between them. For the accessibility implementation for PWA App,  chromium use  NSAccessibilityRemoteUIElement, a private Apple API, to make app shim process have the all accessibility ability of browser process. I doubt the bug has anything to do with NSAccessibilityRemoteUIElement, but I am not familiar with the undocumented API.

See the chromium code(https://source.chromium.org/chromium/chromium/src/+/main:content/browser/accessibility/browser_accessibility_manager_mac.mm;l=171), when pressing the `Space` on checkbox, chromium will call  NSAccessibilityPostNotification with NSAccessibilityValueChangedNotification. Voice Over can’t read the value change for PWA Mac app, but can read for browser scenario.

I use Xcode accessibility inspector tool to see the notification posted from PWA App. I found the notification was sent successfully, but Voice Over can not read it! So I doubt voice over is doing some check to disallow read it in this case?

Do you have any thoughts on this? Appreciate it if there is any comments or response!!

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20230815/b2b3e107/attachment.htm>