[webkit-changes] [WebKit/WebKit] 645818: [Remote Inspection] Element targeting should addit...

[Wenson Hsieh]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 6458185c5c4dad377eeb25a81f4f85bbb4c3db97
      https://github.com/WebKit/WebKit/commit/6458185c5c4dad377eeb25a81f4f85bbb4c3db97
  Author: Wenson Hsieh <wenson_hsieh at apple.com>
  Date:   2024-03-25 (Mon, 25 Mar 2024)

  Changed paths:
    M Source/WebCore/page/ElementTargeting.cpp
    M Source/WebCore/page/ElementTargetingTypes.h
    M Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in
    M Source/WebKit/UIProcess/API/APITargetedElementInfo.h
    M Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm
    M Source/WebKit/UIProcess/API/Cocoa/_WKTargetedElementInfo.h
    M Source/WebKit/UIProcess/API/Cocoa/_WKTargetedElementInfo.mm
    M Source/WebKit/UIProcess/API/Cocoa/_WKTargetedElementRequest.h
    M Source/WebKit/UIProcess/API/Cocoa/_WKTargetedElementRequest.mm
    M Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj
    M Tools/TestWebKitAPI/Tests/WebKitCocoa/ElementTargetingTests.mm
    A Tools/TestWebKitAPI/Tests/WebKitCocoa/element-targeting-2.html

  Log Message:
  -----------
  [Remote Inspection] Element targeting should additionally find nearby out-of-flow elements
https://bugs.webkit.org/show_bug.cgi?id=271616

Reviewed by Abrar Protyasha.

When targeting elements for remote inspection, additionally surface nearby elements which aren't
precisely under the hit test location. Our strategy for this consists of the following:

1.  While collecting targets, aggregate a `Region` containing the rects of all targeted out-of-flow
    elements (and hit-tested elements underneath targeted elements).

2.  After building the list of targets, scan the entire DOM for out-of-flow renderers that are also
    contained in the "nearby targets" region, which also satisfy the same criteria for element
    targeting.

3.  Add these as "nearby targets" to the final list of target infos, to the end of the array.

See below for more details.

Test: ElementTargeting.NearbyOutOfFlowElements

* Source/WebCore/page/ElementTargeting.cpp:
(WebCore::targetedElementInfo):

Refactor this code to pull common logic into lambdas, and implement the steps detailed above.

(WebCore::findTargetedElements):
* Source/WebCore/page/ElementTargetingTypes.h:
* Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in:
* Source/WebKit/UIProcess/API/APITargetedElementInfo.h:
* Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm:
(-[WKWebView _requestTargetedElementInfo:completionHandler:]):
* Source/WebKit/UIProcess/API/Cocoa/_WKTargetedElementInfo.h:

Add a new `isUnderPoint` property, which is `YES` for elements that are directly hit-tested, and
`NO` for nearby targets.

* Source/WebKit/UIProcess/API/Cocoa/_WKTargetedElementInfo.mm:
(-[_WKTargetedElementInfo isUnderPoint]):
* Source/WebKit/UIProcess/API/Cocoa/_WKTargetedElementRequest.h:

Add a new `canIncludeNearbyElements` (default: `YES`) which determines whether or not element
targeting should include elements that have not been hit-tested, but are visually contained within
another element that has been hit-tested.

* Source/WebKit/UIProcess/API/Cocoa/_WKTargetedElementRequest.mm:
(-[_WKTargetedElementRequest init]):
* Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* Tools/TestWebKitAPI/Tests/WebKitCocoa/ElementTargetingTests.mm:
(TestWebKitAPI::TEST):
* Tools/TestWebKitAPI/Tests/WebKitCocoa/element-targeting-2.html: Added.

Add a new API test to exercise the change.

Canonical link: https://commits.webkit.org/276670@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications