[webkit-changes] [WebKit/WebKit] 31c6ba: iOS: Text insertion caret can get out of sync with...

[Tim Horton]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 31c6ba7d4ea89e6add0f115823031e728f092663
      https://github.com/WebKit/WebKit/commit/31c6ba7d4ea89e6add0f115823031e728f092663
  Author: Tim Horton <timothy_horton at apple.com>
  Date:   2022-10-22 (Sat, 22 Oct 2022)

  Changed paths:
    M Source/WebKit/Shared/EditorState.cpp
    M Source/WebKit/Shared/EditorState.h
    M Source/WebKit/Shared/EditorState.serialization.in
    M Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm
    M Source/WebKit/UIProcess/API/glib/WebKitEditorState.cpp
    M Source/WebKit/UIProcess/API/gtk/WebKitWebViewBase.cpp
    M Source/WebKit/UIProcess/API/wpe/WPEView.cpp
    M Source/WebKit/UIProcess/DrawingAreaProxy.h
    M Source/WebKit/UIProcess/RemoteLayerTree/RemoteLayerTreeDrawingAreaProxy.h
    M Source/WebKit/UIProcess/RemoteLayerTree/RemoteLayerTreeDrawingAreaProxy.mm
    M Source/WebKit/UIProcess/WebPageProxy.cpp
    M Source/WebKit/UIProcess/WebPageProxy.h
    M Source/WebKit/UIProcess/ios/WKContentView.mm
    M Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm
    M Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm
    M Source/WebKit/UIProcess/mac/WebViewImpl.mm
    M Source/WebKit/WebProcess/WebPage/Cocoa/WebPageCocoa.mm
    M Source/WebKit/WebProcess/WebPage/WebPage.cpp
    M Source/WebKit/WebProcess/WebPage/WebPage.h
    M Source/WebKit/WebProcess/WebPage/glib/WebPageGLib.cpp
    M Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm
    M Source/WebKit/WebProcess/WebPage/mac/WebPageMac.mm

  Log Message:
  -----------
  iOS: Text insertion caret can get out of sync with the page content
https://bugs.webkit.org/show_bug.cgi?id=244105
rdar://96977291

Reviewed by Wenson Hsieh.

When remote layer tree transactions are slow to flush, intermediate EditorState
updates (from e.g. interpretKeyEvent or other sources) can cause the
UI-process-drawn text insertion caret to jump ahead of the current page tiles.
This is especially egregious when deleting a character, as the insertion
point shows up in between the soon-to-be-deleted character and the previous
character *before* the deleted one disappears.

The problem is exacerbated on slow hardware, or expensive pages.

To fix this, separate "visual" parts of EditorState out into their own
sub-struct, and only update it for EditorState updates that come inside
remote layer tree transactions. Other changes to EditorState will propagate
forward the existing "visual" data, until a new transaction comes in to replace it.

* Source/WebKit/Shared/EditorState.cpp:
(WebKit::EditorState::encode const):
(WebKit::EditorState::decode):
(WebKit::EditorState::PostLayoutData::encode const):
(WebKit::EditorState::PostLayoutData::decode):
(WebKit::EditorState::VisualData::encode const):
(WebKit::EditorState::VisualData::decode):
(WebKit::operator<<):
* Source/WebKit/Shared/EditorState.h:
(WebKit::EditorState::hasPostLayoutData const):
(WebKit::EditorState::ensurePostLayoutData):
(WebKit::EditorState::hasVisualData const):
(WebKit::EditorState::hasPostLayoutAndVisualData const):
(WebKit::EditorState::postLayoutData):
(WebKit::EditorState::postLayoutData const const):
(WebKit::EditorState::visualData):
(WebKit::EditorState::visualData const const):
(WebKit::EditorState::ensureVisualData):
Factor out some geometry into VisualData.
Add hasPostLayoutData and make isMissingPostLayoutData private, to make
the API the same between postLayoutData and visualData.

* Source/WebKit/UIProcess/RemoteLayerTree/RemoteLayerTreeDrawingAreaProxy.mm:
(WebKit::RemoteLayerTreeDrawingAreaProxy::commitLayerTree):
Update EditorState, explicitly allowing VisualData merging.

* Source/WebKit/UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::updateFontAttributesAfterEditorStateChange):
(WebKit::WebPageProxy::updateEditorState):
* Source/WebKit/UIProcess/WebPageProxy.h:
For UI-side compositing clients, make updateEditorState only merge VisualData
changes if they come in a layer tree transaction. Otherwise, we maintain the
existing VisualData. This gets a little tricky, because we need to merge
VisualData changes even if the new EditorState is otherwise stale, and
also need to avoid all of this work (and just take the current incoming
EditorState) for non-UI-side-compositing clients.

* Source/WebKit/WebProcess/WebPage/Cocoa/WebPageCocoa.mm:
(WebKit::WebPage::getPlatformEditorStateCommon const):
* Source/WebKit/WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::editorState const):
Generate VisualData whenever we would generate PostLayoutData.

(WebKit::WebPage::willCommitLayerTree):
(WebKit::WebPage::sendEditorStateUpdate):
(WebKit::WebPage::scheduleFullEditorStateUpdate):
Ensure that if we have scheduled *any* EditorState update since the last transaction,
the next layer tree transaction includes an EditorState update (so that we receive
and merge VisualData).

* Source/WebKit/WebProcess/WebPage/WebPage.h:
* Source/WebKit/WebProcess/WebPage/glib/WebPageGLib.cpp:
(WebKit::WebPage::getPlatformEditorState const):
* Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm:
(WebKit::WebPage::getPlatformEditorState const):
* Source/WebKit/WebProcess/WebPage/mac/WebPageMac.mm:
(WebKit::WebPage::getPlatformEditorState const):
* Source/WebKit/UIProcess/ios/WKContentView.mm:
(-[WKContentView _didCommitLayerTree:]):
* Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm:
(dictionaryRepresentationForEditorState):
* Source/WebKit/UIProcess/API/glib/WebKitEditorState.cpp:
(webkitEditorStateChanged):
* Source/WebKit/UIProcess/API/gtk/WebKitWebViewBase.cpp:
(webkitWebViewBaseUpdateTextInputState):
* Source/WebKit/UIProcess/API/wpe/WPEView.cpp:
(WKWPE::View::selectionDidChange):
* Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm:
(WebKit::WKSelectionDrawingInfo::WKSelectionDrawingInfo):
(-[WKContentView shouldHideSelectionWhenScrolling]):
(-[WKContentView rectToRevealWhenZoomingToFocusedElement]):
(-[WKContentView _selectionClipRect]):
(-[WKContentView canShowNonEmptySelectionView]):
(-[WKContentView webSelectionRects]):
(-[WKContentView _lookupForWebView:]):
(-[WKContentView _shareForWebView:]):
(-[WKContentView _translateForWebView:]):
(-[WKContentView _addShortcutForWebView:]):
(-[WKContentView _cascadeInteractionTintColor]):
(-[WKContentView _showDictionary:]):
(-[WKContentView removeBackgroundMenu]):
(-[WKContentView doAfterComputingImageAnalysisResultsForBackgroundRemoval:]):
(-[WKContentView textFirstRect]):
(-[WKContentView textLastRect]):
(-[WKContentView textInRange:]):
(-[WKContentView selectedTextRange]):
(-[WKContentView markedTextRange]):
(-[WKContentView hasText]):
(-[WKContentView _updateSelectionAssistantSuppressionState]):
(-[WKContentView _selectionChanged]):
(-[WKContentView _updateChangedSelection:]):
* Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm:
(WebKit::WebPageProxy::interpretKeyEvent):
(WebKit::WebPageProxy::dispatchDidUpdateEditorState):
(WebKit::WebPageProxy::selectionBoundingRectInRootViewCoordinates const):
* Source/WebKit/UIProcess/mac/WebViewImpl.mm:
(WebKit::WebViewImpl::selectionDidChange):
(WebKit::WebViewImpl::requestCandidatesForSelectionIfNeeded):
(WebKit::WebViewImpl::handleRequestedCandidates):
(WebKit::WebViewImpl::handleAcceptedCandidate):
(WebKit::WebViewImpl::updateTextTouchBar):
Adopt VisualData everywhere.

Canonical link: https://commits.webkit.org/255884@main