[webkit-changes] [WebKit/WebKit] 35ebce: Avoid triggering image analysis when interrupting ...

[Wenson Hsieh]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 35ebce20fbea85693f15fea9fecda90b08e8c603
      https://github.com/WebKit/WebKit/commit/35ebce20fbea85693f15fea9fecda90b08e8c603
  Author: Wenson Hsieh <wenson_hsieh at apple.com>
  Date:   2023-07-06 (Thu, 06 Jul 2023)

  Changed paths:
    A LayoutTests/fast/images/text-recognition/ios/no-image-analysis-when-interrupting-scrolling-expected.txt
    A LayoutTests/fast/images/text-recognition/ios/no-image-analysis-when-interrupting-scrolling.html
    M LayoutTests/resources/ui-helper.js
    M Source/WebKit/SourcesCocoa.txt
    M Source/WebKit/UIProcess/API/ios/WKWebViewIOS.mm
    M Source/WebKit/UIProcess/Cocoa/MediaPermissionUtilities.mm
    M Source/WebKit/UIProcess/Cocoa/WKStorageAccessAlert.mm
    R Source/WebKit/UIProcess/ios/UIAlertControllerUtilities.h
    R Source/WebKit/UIProcess/ios/UIAlertControllerUtilities.mm
    A Source/WebKit/UIProcess/ios/UIKitUtilities.h
    A Source/WebKit/UIProcess/ios/UIKitUtilities.mm
    M Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm
    M Source/WebKit/UIProcess/ios/WKHighlightLongPressGestureRecognizer.mm
    M Source/WebKit/UIProcess/ios/WKImageAnalysisGestureRecognizer.h
    M Source/WebKit/UIProcess/ios/WKImageAnalysisGestureRecognizer.mm
    M Source/WebKit/UIProcess/ios/WKPasswordView.mm
    M Source/WebKit/UIProcess/ios/WKSyntheticTapGestureRecognizer.mm
    M Source/WebKit/UIProcess/ios/WKWebGeolocationPolicyDeciderIOS.mm
    M Source/WebKit/UIProcess/ios/fullscreen/WKFullScreenViewController.mm
    M Source/WebKit/WebKit.xcodeproj/project.pbxproj
    M Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl
    M Tools/TestRunnerShared/UIScriptContext/UIScriptController.h
    M Tools/WebKitTestRunner/Configurations/WebKitTestRunnerApp.xcconfig
    M Tools/WebKitTestRunner/Configurations/WebKitTestRunnerLibrary.xcconfig
    M Tools/WebKitTestRunner/TestController.h
    M Tools/WebKitTestRunner/cocoa/TestControllerCocoa.mm
    M Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.h
    M Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.mm

  Log Message:
  -----------
  Avoid triggering image analysis when interrupting scroll deceleration by long pressing on an image
https://bugs.webkit.org/show_bug.cgi?id=258929
rdar://111506756

Reviewed by Dean Jackson.

On iOS, when long pressing over an image, the `WKImageAnalysisGestureRecognizer` activates and
begins analyzing images for Live Text and visual look up. This is not desirable for power and
performance in the case where the user is simply trying to scroll through a webpage with many
images, and frequently interrupts scroll deceleration by long pressing on images.

Mitigate this by adding an early return to `-imageAnalysisGestureDidBegin:`, to avoid incurring any
extra work in the case where the gesture is being activated while panning within a scroll view. To
do this, we add a new `lastTouchedScrollView` property on `WKImageAnalysisGestureRecognizer` that
tracks the last `UIScrollView` where touches began. This code is the same as existing logic in
`WKHighlightLongPressGestureRecognizer` and `WKSyntheticTapGestureRecognizer`, but it's somewhat
tricky to share the implementation between these three classes since their superclasses are all
different; for now, we can at least de-duplicate a bit of shared code by adding a new standalone
helper function to find a scroll view for a series of touches in the now-renamed `UIKitUtilities.h`,
and deploying that helper function in these three places.

We also add more testing infrastructure to keep track of a monotonically increasing ID representing
the number of times we trigger VisionKit APIs for image analysis by swizzling out
`VKCImageAnalyzer` methods; see below for more details.

Test: fast/images/text-recognition/ios/no-image-analysis-when-interrupting-scrolling.html

* LayoutTests/fast/images/text-recognition/ios/no-image-analysis-when-interrupting-scrolling-expected.txt: Added.
* LayoutTests/fast/images/text-recognition/ios/no-image-analysis-when-interrupting-scrolling.html: Added.

Add a layout test to verify that interrupting momentum scrolling via a long press doesn't trigger
any additional image analyzer requests in VisionKit.

* LayoutTests/resources/ui-helper.js:
(window.UIHelper.currentImageAnalysisRequestID):

Add a helper method to return the current (monotonically increasing) image analysis request ID.

* Source/WebKit/SourcesCocoa.txt:
* Source/WebKit/UIProcess/API/ios/WKWebViewIOS.mm:
* Source/WebKit/UIProcess/Cocoa/MediaPermissionUtilities.mm:
* Source/WebKit/UIProcess/Cocoa/WKStorageAccessAlert.mm:

Adjust `UIAlertControllerUtilities.h` -> `UIKitUtilities.h`; see below.

* Source/WebKit/UIProcess/ios/UIKitUtilities.h: Renamed from Source/WebKit/UIProcess/ios/UIAlertControllerUtilities.h.
* Source/WebKit/UIProcess/ios/UIKitUtilities.mm: Renamed from Source/WebKit/UIProcess/ios/UIAlertControllerUtilities.mm.

Add a new helper method to this file, to return a `UIScrollView` corresponding to a set of `UITouch`
objects; also rename `UIAlertControllerUtilities.*` to `UIKitUtilities.*`, since it now contains
helper functions related to UIKit (not just `UIAlertController`).

(WebKit::createUIAlertController):
(WebKit::scrollViewForTouches):
* Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm:
(-[WKContentView imageAnalysisGestureDidBegin:]):

Implement the actual fix here by checking if the `WKImageAnalysisGestureRecognizer`'s last touched
scroll view is being panned.

* Source/WebKit/UIProcess/ios/WKHighlightLongPressGestureRecognizer.mm:
(-[WKHighlightLongPressGestureRecognizer touchesBegan:withEvent:]):

Deduplicate some code, using the new helper function in `UIKitUtilities.h`.

* Source/WebKit/UIProcess/ios/WKImageAnalysisGestureRecognizer.h:
* Source/WebKit/UIProcess/ios/WKImageAnalysisGestureRecognizer.mm:
(-[WKImageAnalysisGestureRecognizer reset]):
(-[WKImageAnalysisGestureRecognizer touchesBegan:withEvent:]):

Implement `-lastTouchedScrollView` on the image analysis gesture.

* Source/WebKit/UIProcess/ios/WKPasswordView.mm:

Rename `UIAlertControllerUtilities.h` -> `UIKitUtilities.h` (see above).

* Source/WebKit/UIProcess/ios/WKSyntheticTapGestureRecognizer.mm:
(-[WKSyntheticTapGestureRecognizer touchesBegan:withEvent:]):

Deduplicate some more code, using the new helper function in `UIKitUtilities.h`.

* Source/WebKit/UIProcess/ios/WKWebGeolocationPolicyDeciderIOS.mm:
* Source/WebKit/UIProcess/ios/fullscreen/WKFullScreenViewController.mm:

Rename `UIAlertControllerUtilities.h` -> `UIKitUtilities.h` (see above).

* Source/WebKit/WebKit.xcodeproj/project.pbxproj:
* Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl:

Add a new script controller hook to return the number of times we triggered image analyzer requests.

* Tools/TestRunnerShared/UIScriptContext/UIScriptController.h:
(WTR::UIScriptController::currentImageAnalysisRequestID const):
* Tools/WebKitTestRunner/Configurations/WebKitTestRunnerApp.xcconfig:
* Tools/WebKitTestRunner/Configurations/WebKitTestRunnerLibrary.xcconfig:

Link against PAL, so that we can use extant soft-linking macros in `VisionKitCoreSoftLink.h`.

* Tools/WebKitTestRunner/TestController.h:
* Tools/WebKitTestRunner/cocoa/TestControllerCocoa.mm:
(swizzledProcessImageAnalysisRequest):

Swizzle out calls to `-processRequest:progressHandler:completionHandler:` to always return a
consistent result (error), and increment the global `gCurrentImageAnalysisRequestID` count returned
by the new script controller attribute.

(WTR::TestController::cocoaPlatformInitialize):
(WTR::TestController::currentImageAnalysisRequestID):
* Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.h:
* Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.mm:
(WTR::UIScriptControllerCocoa::currentImageAnalysisRequestID const):

Canonical link: https://commits.webkit.org/265824@main