[webkit-changes] [WebKit/WebKit] 48c8c3: [iOS] Image analysis menu items for images with ma...

[Wenson Hsieh]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 48c8c316241e1519cd2deb1dcdda00788fa629a6
      https://github.com/WebKit/WebKit/commit/48c8c316241e1519cd2deb1dcdda00788fa629a6
  Author: Wenson Hsieh <wenson_hsieh at apple.com>
  Date:   2023-11-29 (Wed, 29 Nov 2023)

  Changed paths:
    A LayoutTests/fast/events/touch/ios/long-press-on-image-with-machine-readable-code-menu-items-expected.txt
    A LayoutTests/fast/events/touch/ios/long-press-on-image-with-machine-readable-code-menu-items.html
    M LayoutTests/resources/ui-helper.js
    M Source/WebKit/UIProcess/ios/WKContentViewInteraction.h
    M Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm
    M Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl
    M Tools/TestRunnerShared/UIScriptContext/UIScriptController.h
    M Tools/WebKitTestRunner/TestController.h
    M Tools/WebKitTestRunner/cocoa/TestControllerCocoa.mm
    M Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.h
    M Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.mm

  Log Message:
  -----------
  [iOS] Image analysis menu items for images with machine-readable codes sometimes fail to appear
https://bugs.webkit.org/show_bug.cgi?id=265544
rdar://118540193

Reviewed by Megan Gardner.

Our current logic for dynamically inserting image analysis menu items (such as "Look Up" and any
menu items for machine-readable codes) works like this:

1.  We kick off image analysis for machine-readable codes and visual search when the context menu is
    about to be presented. The menu is presented with a hidden placeholder menu item, which
    represents the set of dynamically-inserted image analysis items we might insert in the future.

2.  When the image analysis started by (1) is done, we use `-updateVisibleMenuWithBlock:` on the
    context menu interaction to replace the placeholder item above with the final set of image
    analysis items (only if the WebKit client didn't remove the placeholder menu item).

However, there's a subtle race here, where image analysis may complete before the context menu has a
visible menu. In this scenario, the call to `-updateVisibleMenuWithBlock:` returns early without
calling the update block, and we end up missing our only chance to swap out the placeholder item for
the actual image analysis items.

To fix this, we detect the case where our image analysis completes before the menu is visible, and
try again in `-contextMenuInteraction:willDisplayMenuForConfiguration:animator:`, after the menu
is guaranteed to be visible.

Test: fast/events/touch/ios/long-press-on-image-with-machine-readable-code-menu-items.html

* LayoutTests/fast/events/touch/ios/long-press-on-image-with-machine-readable-code-menu-items-expected.txt: Added.
* LayoutTests/fast/events/touch/ios/long-press-on-image-with-machine-readable-code-menu-items.html: Added.

Add a new layout test, along with some additional test infrastructure to exercise this fix.

* LayoutTests/resources/ui-helper.js:

Add a way to simulate MRC menu item results when long-pressing on an image in layout tests, on iOS.

(window.UIHelper.contextMenuItems):
(window.UIHelper.installFakeMachineReadableCodeResultsForImageAnalysis):
* Source/WebKit/UIProcess/ios/WKContentViewInteraction.h:
* Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm:
(-[WKContentView _setUpImageAnalysis]):
(-[WKContentView _tearDownImageAnalysis]):
(-[WKContentView imageAnalysisGestureDidBegin:]):

Replace the extant boolean flag `_waitingForDynamicImageAnalysisContextMenuActions` with a tri-state
enum type, `_dynamicImageAnalysisContextMenuState`, which allows us to appropriately deal with the
corner case where image analysis completes before the context menu has any visible items. See above
for more details.

(-[WKContentView _completeImageAnalysisRequestForContextMenu:requestIdentifier:hasTextResults:]):
(-[WKContentView _insertDynamicImageAnalysisContextMenuItemsIfPossible]):

Pull the existing logic for updating visible context menu items to include the dynamically-inserted
image analysis items into a separate helper method, so that we can invoke it after the context menu
is done presenting.

If the update block runs successfully (i.e. the menu is visible), then we transition directly to the
`NotWaiting` state and we're done. However, if the update block never runs when calling into
`-updateVisibleMenuWithBlock:`, we instead transition to `WaitingForVisibleMenu` and wait until the
menu is done presenting, before trying to perform the update again.

(-[WKContentView placeholderForDynamicallyInsertedImageAnalysisActions]):
(-[WKContentView contextMenuInteraction:willDisplayMenuForConfiguration:animator:]):

Update the menu to include MRC menu items (and other dynamically-inserted image analysis items) if
needed — note that we'll only perform the update here, in the case where we failed to run the update
block at all, in `-_completeImageAnalysisRequestForContextMenu:requestIdentifier:hasTextResults:`.

* Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl:
* Tools/TestRunnerShared/UIScriptContext/UIScriptController.h:
(WTR::UIScriptController::installFakeMachineReadableCodeResultsForImageAnalysis):
* Tools/WebKitTestRunner/TestController.h:
* Tools/WebKitTestRunner/cocoa/TestControllerCocoa.mm:
(fakeMachineReadableCodeMenuForTesting):
(-[FakeMachineReadableCodeImageAnalysis allLines]):
(-[FakeMachineReadableCodeImageAnalysis hasResultsForAnalysisTypes:]):
(-[FakeMachineReadableCodeImageAnalysis mrcMenu]):

Add a helper class that represents a fake VisionKit image analysis result that contains MRC menu
items (for simplicity, there's only one hard-coded item titled "QR code").

(swizzledProcessImageAnalysisRequest):
(WTR::TestController::installFakeMachineReadableCodeResultsForImageAnalysis):
(WTR::TestController::shouldUseFakeMachineReadableCodeResultsForImageAnalysis const):
(WTR::TestController::cocoaResetStateToConsistentValues):
* Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.h:
* Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.mm:
(WTR::UIScriptControllerCocoa::installFakeMachineReadableCodeResultsForImageAnalysis):

Canonical link: https://commits.webkit.org/271317@main