[webkit-changes] [WebKit/WebKit] edca53: [Remote Inspection] Add the ability to search for ...

[Wenson Hsieh]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: edca530205c5e76991b0cf709322fb121f8b575d
      https://github.com/WebKit/WebKit/commit/edca530205c5e76991b0cf709322fb121f8b575d
  Author: Wenson Hsieh <wenson_hsieh at apple.com>
  Date:   2024-04-25 (Thu, 25 Apr 2024)

  Changed paths:
    M Source/WebCore/page/ElementTargetingController.cpp
    M Source/WebCore/page/ElementTargetingController.h
    M Source/WebCore/page/ElementTargetingTypes.h
    M Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in
    M Source/WebKit/UIProcess/API/APITargetedElementInfo.h
    M Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm
    M Source/WebKit/UIProcess/API/Cocoa/_WKTargetedElementInfo.h
    M Source/WebKit/UIProcess/API/Cocoa/_WKTargetedElementInfo.mm
    M Source/WebKit/UIProcess/API/Cocoa/_WKTargetedElementRequest.h
    M Source/WebKit/UIProcess/API/Cocoa/_WKTargetedElementRequest.mm
    M Tools/TestWebKitAPI/Tests/WebKitCocoa/ElementTargetingTests.mm
    M Tools/TestWebKitAPI/Tests/WebKitCocoa/element-targeting-7.html
    M Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.mm

  Log Message:
  -----------
  [Remote Inspection] Add the ability to search for target elements by text
https://bugs.webkit.org/show_bug.cgi?id=273222
rdar://126991300

Reviewed by Abrar Protyasha and Aditya Keerthi.

Add support for:

1.  A `-searchableText` property on `_WKTargetedElementInfo`, which exposes the first 100 characters
    of the longest contiguous text run in the targeted element.

2.  A way to initialize `_WKTargetedElementRequest` with a `searchText` string instead of a `point`,
    which retrieves a targeted element by searching in the document (and recursively in any
    subframes) for the first range that matches the text.

See below for more details.

* Source/WebCore/page/ElementTargetingController.cpp:
(WebCore::collectDocumentElementsFromChildFrames):
(WebCore::searchableTextForTarget):
(WebCore::targetedElementInfo):
(WebCore::ElementTargetingController::findTargets):
(WebCore::ElementTargetingController::findNodes):

Refactor the element targeting logic to work with both hit-testing (based on a point in root view
space) or text search (based on a search string). To do this, we split the logic out into two parts:

1.  The first part computes a list of potential candidates (`Vector<Ref<Node>>`), using one of two
    strategies: hit-testing or text search.

2.  The second part takes the list of potential candiates from (1) and resolves the final list of
    targeted elements.

The new text search codepath hooks into the existing targeting code by searching the DOM for ranges
that match the given text and surfacing the ancestor chain as potential candidates, which the rest
of the targeting heuristic pipeline consumes.

(WebCore::searchForElementContainingText):
(WebCore::ElementTargetingController::extractTargets):
* Source/WebCore/page/ElementTargetingController.h:
* Source/WebCore/page/ElementTargetingTypes.h:
* Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in:

Back `TargetedElementRequest` with a variant of `FloatPoint, String` instead of just a `FloatPoint`.
The `FloatPoint` represents a hit-testing location in root view coordinates, while the `String`
represents a search string.

* Source/WebKit/UIProcess/API/APITargetedElementInfo.h:
* Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm:
(-[WKWebView _requestTargetedElementInfo:completionHandler:]):
* Source/WebKit/UIProcess/API/Cocoa/_WKTargetedElementInfo.h:
* Source/WebKit/UIProcess/API/Cocoa/_WKTargetedElementInfo.mm:
(-[_WKTargetedElementInfo searchableText]):
(-[_WKTargetedElementInfo isNearbyTarget]):
(-[_WKTargetedElementInfo isUnderPoint]): Deleted.

Rename this property to `isNearbyTarget`, since `isUnderPoint` doesn't make sense in the case where
the targeted element was requested via text search.

* Source/WebKit/UIProcess/API/Cocoa/_WKTargetedElementRequest.h:
* Source/WebKit/UIProcess/API/Cocoa/_WKTargetedElementRequest.mm:
(-[_WKTargetedElementRequest initWithSearchText:]):
(-[_WKTargetedElementRequest initWithPoint:]):
(-[_WKTargetedElementRequest searchText]):
(-[_WKTargetedElementRequest init]): Deleted.

Instead of having clients use `-init` and then set a point on the request, we add initializer
methods to take either a string (for the search text) or a point in web view coordinates, and then
expose the point or search text as readonly properties only. (For now, we need to keep exposing
`-setPoint:` for source compatibility, but this should become readonly shortly).

* Tools/TestWebKitAPI/Tests/WebKitCocoa/ElementTargetingTests.mm:
(-[WKWebView _targetedElementInfo:]):
(-[WKWebView targetedElementInfoAt:]):
(-[WKWebView targetedElementInfoWithText:]):
(TestWebKitAPI::TEST(ElementTargeting, NearbyOutOfFlowElements)):
(TestWebKitAPI::TEST(ElementTargeting, RequestTargetedElementsBySearchableText)):

Add a new API test to exercise both SPIs.

* Tools/TestWebKitAPI/Tests/WebKitCocoa/element-targeting-7.html:
* Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.mm:
(WTR::UIScriptControllerCocoa::requestRenderedTextForFrontmostTarget):
(WTR::UIScriptControllerCocoa::adjustVisibilityForFrontmostTarget):

Canonical link: https://commits.webkit.org/278027@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications