[webkit-changes] [WebKit/WebKit] 8af90c: [Remote Inspection] Make the repeated element targ...

[Wenson Hsieh]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 8af90c08dbd9df112cb58f4ed043180dc8c7d512
      https://github.com/WebKit/WebKit/commit/8af90c08dbd9df112cb58f4ed043180dc8c7d512
  Author: Wenson Hsieh <wenson_hsieh at apple.com>
  Date:   2024-03-28 (Thu, 28 Mar 2024)

  Changed paths:
    M Source/WebCore/page/ElementTargetingController.cpp
    M Source/WebCore/page/ElementTargetingController.h
    M Source/WebKit/WebProcess/WebPage/WebPage.cpp
    M Tools/TestWebKitAPI/PlatformUtilities.h
    M Tools/TestWebKitAPI/Tests/WebKitCocoa/ElementTargetingTests.mm
    M Tools/TestWebKitAPI/Tests/WebKitCocoa/GPUProcess.mm
    M Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebViewSnapshot.mm
    M Tools/TestWebKitAPI/Tests/WebKitCocoa/element-targeting-2.html
    M Tools/TestWebKitAPI/cocoa/PlatformUtilitiesCocoa.mm
    M Tools/TestWebKitAPI/cocoa/TestWKWebView.h
    M Tools/TestWebKitAPI/cocoa/TestWKWebView.mm

  Log Message:
  -----------
  [Remote Inspection] Make the repeated element targeting heuristic robust in the case where elements are unparented
https://bugs.webkit.org/show_bug.cgi?id=271848

Reviewed by Aditya Keerthi and Abrar Rahman Protyasha.

Currently, client rects for visibility adjustment are added at visibility adjustment time, based on
the current client rect of the target element. This has a major drawback, in that a target could be
unparented (or otherwise prematurely hidden) by the page, and we'll fail to add any client rects
corresponding to the target element.

Instead, this patch refactors that mechanism to be more robust, by computing and caching the last
set of targeted element client rects up front when finding targets. Later, when we actually go and
apply visibility adjustment, we'll consult these cached target rects instead of reading the current
element state.

Test: ElementTargeting.AdjustVisibilityForUnparentedElement

* Source/WebCore/page/ElementTargetingController.cpp:
(WebCore::computeClientRect):

Move this static helper function up in this file.

(WebCore::ElementTargetingController::findTargets):
(WebCore::ElementTargetingController::adjustVisibility):

Also refactor this logic to update adjustment regions, such that it happens before any of the
targeted elements are hidden. This is possible now that we only rely on the element identifiers and
`m_pendingAdjustmentClientRects` (from the last element targeting call).

(WebCore::ElementTargetingController::resetAdjustmentRegions):
* Source/WebCore/page/ElementTargetingController.h:

Add `m_pendingAdjustmentClientRects`, which tracks the last set of targeted elements which might be
imminently hidden. Keeping around only this recent state helps avoided unbounded growth as the user
targets more elements around the page, but still allows us to handle this corner case where targeted
elements can no longer be hidden.

* Source/WebKit/WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::adjustVisibilityForTargetedElements):

Adjust the WebCore method to take a list of element identifiers instead of elements; see above for
more details.

* Tools/TestWebKitAPI/PlatformUtilities.h:
* Tools/TestWebKitAPI/Tests/WebKitCocoa/ElementTargetingTests.mm:
(-[WKWebView adjustVisibilityForTargets:]):

Add a new API test to exercise the change.

(TestWebKitAPI::TEST(ElementTargeting, AdjustVisibilityForUnparentedElement)):
* Tools/TestWebKitAPI/Tests/WebKitCocoa/GPUProcess.mm:
(TestWebKitAPI::TEST(GPUProcess, CanvasBasicCrashHandling)):
(TestWebKitAPI::convertToCGImage): Deleted.
* Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebViewSnapshot.mm:
(-[TestSnapshotWrapper takeSnapshotWithWebView:configuration:completionHandler:]):
(TestWebKitAPI::TEST(WKWebView, SnapshotImageError)):
(TestWebKitAPI::TEST(WKWebView, SnapshotImageEmptyRect)):
(TestWebKitAPI::TEST(WKWebView, SnapshotImageZeroWidth)):
(TestWebKitAPI::TEST(WKWebView, SnapshotImageZeroSizeView)):
(TestWebKitAPI::TEST(WKWebView, SnapshotImageZeroSizeViewNoConfiguration)):
(TestWebKitAPI::TEST(WKWebView, SnapshotImageBaseCase)):
(TestWebKitAPI::TEST(WKWebView, SnapshotImageScale)):
(TestWebKitAPI::TEST(WKWebView, SnapshotImageNilConfiguration)):
(TestWebKitAPI::TEST(WKWebView, SnapshotImageUninitializedConfiguration)):
(TestWebKitAPI::TEST(WKWebView, SnapshotImageUninitializedSnapshotWidth)):
(TestWebKitAPI::TEST(WKWebView, SnapshotImageLargeAsyncDecoding)):
(TestWebKitAPI::TEST(WKWebView, SnapshotAfterScreenUpdates)):
(TestWebKitAPI::TEST(WKWebView, SnapshotWithoutAfterScreenUpdates)):
(TestWebKitAPI::TEST(WKWebView, SnapshotWebGL)):
(TestWebKitAPI::TEST(WKWebView, SnapshotWithoutSelectionHighlighting)):
(TestWebKitAPI::TEST(WKWebView, SnapshotWithContentsRect)):

Refactor some existing API tests to use the new `CGImage` conversion helpers.

(convertToCGImage): Deleted.
(TEST(WKWebView, SnapshotImageError)): Deleted.
(TEST(WKWebView, SnapshotImageEmptyRect)): Deleted.
(TEST(WKWebView, SnapshotImageZeroWidth)): Deleted.
(TEST(WKWebView, SnapshotImageZeroSizeView)): Deleted.
(TEST(WKWebView, SnapshotImageZeroSizeViewNoConfiguration)): Deleted.
(TEST(WKWebView, SnapshotImageEmptyWithOutOfScopeCompletionHandler)): Deleted.
(TEST(WKWebView, SnapshotImageBaseCase)): Deleted.
(TEST(WKWebView, SnapshotImageScale)): Deleted.
(TEST(WKWebView, SnapshotImageNilConfiguration)): Deleted.
(TEST(WKWebView, SnapshotImageUninitializedConfiguration)): Deleted.
(TEST(WKWebView, SnapshotImageUninitializedSnapshotWidth)): Deleted.
(TEST(WKWebView, SnapshotImageLargeAsyncDecoding)): Deleted.
(TEST(WKWebView, SnapshotAfterScreenUpdates)): Deleted.
(TEST(WKWebView, SnapshotWithoutAfterScreenUpdates)): Deleted.
(TEST(WKWebView, SnapshotWebGL)): Deleted.
(TEST(WKWebView, SnapshotWithoutSelectionHighlighting)): Deleted.
(TEST(WKWebView, SnapshotWithContentsRect)): Deleted.

Refactor some existing API tests to use the new `CGImage` conversion helpers.

* Tools/TestWebKitAPI/Tests/WebKitCocoa/element-targeting-2.html:
* Tools/TestWebKitAPI/cocoa/PlatformUtilitiesCocoa.mm:
(TestWebKitAPI::Util::convertToCGImage):

Pull out some helper functions to convert platform `NSImage` / `UIImage` into `CGImage`.

* Tools/TestWebKitAPI/cocoa/TestWKWebView.h:
* Tools/TestWebKitAPI/cocoa/TestWKWebView.mm:
(-[WKWebView snapshotAfterScreenUpdates]):

Add a new helper method for testing, that grabs a snapshot of the web view (after screen updates) as
a `CGImage`.

Canonical link: https://commits.webkit.org/276824@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications