[Webkit-unassigned] [Bug 278458] New: AX: Allow strict replacements via AXTextOperation

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=278458

            Bug ID: 278458
           Summary: AX: Allow strict replacements via AXTextOperation
           Product: WebKit
           Version: Safari 18
          Hardware: All
                OS: macOS 15
            Status: NEW
          Severity: Enhancement
          Priority: P2
         Component: Accessibility
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: samar.sunkaria at grammarly.com
                CC: andresg_22 at apple.com,
                    webkit-bug-importer at group.apple.com

The AXTextOperation accessibility API for macOS allows text replacement within a specified list of text marker ranges, among other operations. This AX API is particularly beneficial for clients like Grammarly for macOS, that utilize the accessibility subsystem to perform text replacements. However, in its current form, AXTextOperation arbitrarily changes the case of the replacement string and performs a “smart replacement” that adds spaces around the replaced text, severely limiting the usability of the API. We propose the following modifications to the API to allow better control over the replacement. While the same outcome can be achieved by chaining multiple calls to AX APIs/emitting CGEvents, AXTextOperation allows for a more efficient implementation.

I can submit a patch if this is deemed to be an acceptable change to AXTextOperation.

Preserving the Case of the Replacement String:

`TextOperationReplace` modifies the case of the replacement string supplied to the API, changing it to either capitalized or lowercase, presumably to match the case of the text being replaced. As a result, clients of AXTextOperation cannot enforce the case of the replacement string they provide.

A real example of the existing behavior that makes incorrect/unexpected changes shows up when the case of the replacement string must be preserved, e.g. when replacing “Mac OS X” with “macOS”. The current implementation would infer that “Mac OS X” is capitalized and would attempt to capitalize the replacement string “macOS” -> “MacOS” to preserve the case. 

AXTextOperation would benefit from extending the API to allow for text replacements that preserve the case of the replacement string for clients that know/want the replacement string to maintain its case. This can be achieved by adding an additional case to `AXTextOperationType`, e.g. `TextOperationReplacePreserveCase`. When supplying the new type, the replacement string will be used as is, without attempting to match its case to the string being replaced. Since this is an additive change, the existing clients of AXTextOperation will remain unaffected.

Allow Disabling Smart Replacements:

All AXTextOperation types that replace text (i.e. all operations types except “select”), always perform a “smart replace”, which assumes that the replacement string is not replacing partial words and attempts to add spaces around it. While this is an excellent feature for certain replacements, it can severely limit the usability of AXTextOperation for partial word replacements.

For example, when replacing “MacOS” with “macOS”, we could attempt to perform a minimal change, where the “M” is replaced with “m” (given we already have `TextOperationReplacePreserveCase` from above). The current implementation would perform a smart replacement and, therefore, add a space after the replaced “m”, “MacOS” -> “m acOS”.

In the same vein as the proposed change for preserving the case, extending AXTextOperation to support _non-smart_ replacements would be a great addition to the API for clients that want to perform a partial word replacement. This can be achieved by adding a key to the AXTextOperation dictionary for disabling smart replacement, e.g. “AXTextOperationSmartReplace”. The value of this key would be a boolean, where NO/false disables smart replacements. Since this is an additive change, the existing clients of AXTextOperation will remain unaffected.

Usage:

Client pseudo-code below:
```
// Replace “Mac OS X” -> “macOS”
CFTypeRef result;
AXUIElementCopyParameterizedAttributeValue(element, "AXTextOperation", @[
  NSAccessibilityTextOperationType: NSAccessibilityTextOperationReplacePreserveCase,
  NSAccessibilityTextOperationReplacementString: @"macOS",
  NSAccessibilityTextOperationMarkerRanges: @[ rangeOfMacOSX ]
], &result)

// Replace "MacOS" -> "macOS", by only replacing the first character
CFTypeRef result;
AXUIElementCopyParameterizedAttributeValue(element, "AXTextOperation", @[
  NSAccessibilityTextOperationType: NSAccessibilityTextOperationReplacePreserveCase,
  NSAccessibilityTextOperationReplacementString: @"m",
  NSAccessibilityTextOperationSmartReplace: kCFBooleanFalse,
  NSAccessibilityTextOperationMarkerRanges: @[ rangeOfM ]
], &result)
```

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20240821/350f6844/attachment.htm>