[webkit-changes] [WebKit/WebKit] a364c0: [visionOS] Scrolling sometimes drops frames due to...

[Wenson Hsieh]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: a364c0f2e2f3a18524009049a2728a37ab764574
      https://github.com/WebKit/WebKit/commit/a364c0f2e2f3a18524009049a2728a37ab764574
  Author: Wenson Hsieh <wenson_hsieh at apple.com>
  Date:   2025-02-20 (Thu, 20 Feb 2025)

  Changed paths:
    M Source/WebCore/editing/cocoa/AttributedString.mm
    M Source/WebCore/editing/cocoa/WebCoreTextAttachment.h
    M Source/WebCore/editing/cocoa/WebCoreTextAttachment.mm
    M Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm
    M Tools/TestWebKitAPI/Tests/WebKitCocoa/DocumentEditingContext.mm

  Log Message:
  -----------
  [visionOS] Scrolling sometimes drops frames due to spatial document editing context requests
https://bugs.webkit.org/show_bug.cgi?id=288039
rdar://144889554

Reviewed by Tim Horton and Abrar Rahman Protyasha.

On some configurations of visionOS, UIKit (through RemoteTextInput) requests spatial document
editing contexts with attributed strings, whenever the web view's visible content bounds change
(e.g. while scrolling); the spatial rect passed in is simply the bounds of the web view (i.e. an
approximation of the visible content rect). While a bit excessive, this is made even worse by two
latent issues in WebKit's logic for processing spatial document editing contexts:

1.  We currently (naively) hit-test the top left and bottom right corners of the passed-in rect, and
    expand the context range to include all content in between. In the case where we hit-test to an
    out-of-flow container (e.g. popup or banner), however, this often leads to the `rangeOfInterest`
    encompassing the entire contents of the webpage, which defeats the purpose of spatially
    constraining the document context.

    To fix this, we devise a new heuristic that clamps these hit-tested positions to a range around
    the current selection, whose endpoints fall within the requested spatial rect. To achieve this,
    we iterate outwards from the current selection's endpoints by line, and stop once we hit a
    visible position that does not intersect with the given rect.

2.  The web content process does not have a connection to the windowserver, so calls into
    `+[UIImage imageNamed:inBundle:withConfiguration:]` which indirectly initialize the main
    `UIScreen` in order to read the current trait collection fail after hanging for 5 seconds. When
    creating attributed strings, we end up exercising this codepath via `extractDictionary`, because
    we try to check whether the text attachment's image is equal to the bundle's missing attachment
    image:

    ```
        if ([value isKindOfClass:PlatformNSTextAttachment]) {
        if ([value image] == webCoreTextAttachmentMissingPlatformImage())
            return { { TextAttachmentMissingImage() } };
    ```

    Work around this for now by adding a separate `isWebCoreTextAttachmentMissingPlatformImage`
    helper function that takes an image and returns whether or not the image is equal to the
    missing attachment image, crucially *without* forcing the image to be initialized if it has not
    already been.

* Source/WebCore/editing/cocoa/AttributedString.mm:
(WebCore::extractValue):

Fix (2) above by adopting `isWebCoreTextAttachmentMissingPlatformImage`.

* Source/WebCore/editing/cocoa/WebCoreTextAttachment.h:
* Source/WebCore/editing/cocoa/WebCoreTextAttachment.mm:
(WebCore::webCoreTextAttachmentMissingPlatformImageIfExists):
(WebCore::webCoreTextAttachmentMissingPlatformImage):
(WebCore::isWebCoreTextAttachmentMissingPlatformImage):

Add a very cheap way to check if the given image is a missing text attachment, in the case where a
text attachment missing image hasn't been initialized yet.

(webCoreTextAttachmentMissingPlatformImage): Deleted.
* Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm:
(WebKit::WebPage::requestDocumentEditingContext):

Fix (1) above.

* Tools/TestWebKitAPI/Tests/WebKitCocoa/DocumentEditingContext.mm:
(TEST(DocumentEditingContext, SpatialAndCurrentSelectionRequest_LimitContextToVisibleText)):

Add an API test to exercise the change.

Canonical link: https://commits.webkit.org/290710@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications