[webkit-changes] [WebKit/WebKit] 3d7827: Add a way for internal WebKit clients to extract a...

[Wenson Hsieh]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 3d7827fa1ae86043d77a65d3fe8479ba5b2ee20c
      https://github.com/WebKit/WebKit/commit/3d7827fa1ae86043d77a65d3fe8479ba5b2ee20c
  Author: Wenson Hsieh <wenson_hsieh at apple.com>
  Date:   2024-10-25 (Fri, 25 Oct 2024)

  Changed paths:
    M Source/WebCore/page/text-extraction/TextExtraction.cpp
    M Source/WebCore/page/text-extraction/TextExtraction.h
    M Source/WebKit/Shared/API/APIObject.h
    M Source/WebKit/Shared/API/Cocoa/WebKitPrivate.h
    M Source/WebKit/Shared/Cocoa/APIObject.mm
    M Source/WebKit/Sources.txt
    M Source/WebKit/SourcesCocoa.txt
    A Source/WebKit/UIProcess/API/APITextRun.cpp
    A Source/WebKit/UIProcess/API/APITextRun.h
    M Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm
    M Source/WebKit/UIProcess/API/Cocoa/WKWebViewPrivate.h
    A Source/WebKit/UIProcess/API/Cocoa/_WKTextRun.h
    A Source/WebKit/UIProcess/API/Cocoa/_WKTextRun.mm
    A Source/WebKit/UIProcess/API/Cocoa/_WKTextRunInternal.h
    M Source/WebKit/UIProcess/WebPageProxy.cpp
    M Source/WebKit/UIProcess/WebPageProxy.h
    M Source/WebKit/WebKit.xcodeproj/project.pbxproj
    M Source/WebKit/WebProcess/WebPage/WebPage.cpp
    M Source/WebKit/WebProcess/WebPage/WebPage.h
    M Source/WebKit/WebProcess/WebPage/WebPage.messages.in
    M Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj
    M Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebViewGetContents.mm
    M Tools/TestWebKitAPI/Tests/WebKitCocoa/nested-frames.html
    A Tools/TestWebKitAPI/Tests/WebKitCocoa/subframes.html

  Log Message:
  -----------
  Add a way for internal WebKit clients to extract a list of all text and text rects on a webpage
https://bugs.webkit.org/show_bug.cgi?id=281795
rdar://138221056

Reviewed by Abrar Rahman Protyasha.

Add experimental WebKit SPI to collect all of the text on a webpage, along with bounding rects in
`WKWebView` coordinates. See below for more details.

* Source/WebCore/page/text-extraction/TextExtraction.cpp:
(WebCore::TextExtraction::extractAllTextAndRectsRecursive):
(WebCore::TextExtraction::extractAllTextAndRects):

Implement the main logic here, which uses `TextIterator` to recursively extract visible text content
from each frame on the page.

* Source/WebCore/page/text-extraction/TextExtraction.h:
* Source/WebKit/Shared/API/APIObject.h:
* Source/WebKit/Shared/API/Cocoa/WebKitPrivate.h:
* Source/WebKit/Shared/Cocoa/APIObject.mm:
(API::Object::newObject):
* Source/WebKit/Sources.txt:
* Source/WebKit/SourcesCocoa.txt:
* Source/WebKit/UIProcess/API/APITextRun.cpp: Copied from Source/WebCore/page/text-extraction/TextExtraction.h.
(API::TextRun::rectInWebView const):
* Source/WebKit/UIProcess/API/APITextRun.h: Copied from Source/WebCore/page/text-extraction/TextExtraction.h.

Add a new `API::Object` that represents a single piece of visible text, along with a rect in root
view coordinates.

* Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm:
(-[WKWebView _requestAllTextWithCompletionHandler:]):
* Source/WebKit/UIProcess/API/Cocoa/WKWebViewPrivate.h:
* Source/WebKit/UIProcess/API/Cocoa/_WKTextRun.h: Copied from Source/WebCore/page/text-extraction/TextExtraction.h.
* Source/WebKit/UIProcess/API/Cocoa/_WKTextRun.mm: Copied from Source/WebCore/page/text-extraction/TextExtraction.h.
(-[_WKTextRun dealloc]):
(-[_WKTextRun _apiObject]):
(-[_WKTextRun text]):
(-[_WKTextRun rectInWebView]):
(-[_WKTextRun debugDescription]):

Add a new Objective-C object, that wraps the C++ `API::Object` above.

* Source/WebKit/UIProcess/API/Cocoa/_WKTextRunInternal.h: Copied from Source/WebCore/page/text-extraction/TextExtraction.h.
* Source/WebKit/UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::requestAllTextAndRects):
* Source/WebKit/UIProcess/WebPageProxy.h:
* Source/WebKit/WebKit.xcodeproj/project.pbxproj:
* Source/WebKit/WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::requestAllTextAndRects):
* Source/WebKit/WebProcess/WebPage/WebPage.h:
* Source/WebKit/WebProcess/WebPage/WebPage.messages.in:
* Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebViewGetContents.mm:

Drive-by fix: wrap all of these tests in `namespace TestWebKitAPI`.

(-[WKWebView _allTextRuns]):
(TestWebKitAPI::TEST(WKWebView, GetContentsShouldReturnString)):
(TestWebKitAPI::TEST(WKWebView, GetContentsShouldFailWhenClosingPage)):
(TestWebKitAPI::TEST(WKWebView, GetContentsOfAllFramesShouldReturnString)):
(TestWebKitAPI::TEST(WKWebView, GetContentsShouldReturnAttributedString)):
(TestWebKitAPI::TEST(WKWebView, GetContentsWithOpticallySizedFontShouldReturnAttributedString)):
(TestWebKitAPI::TEST(WKWebView, AttributedStringAccessibilityLabel)):
(TestWebKitAPI::TEST(WKWebView, AttributedStringAttributeTypes)):
(TestWebKitAPI::TEST(WKWebView, AttributedStringWithoutNetworkLoads)):
(TestWebKitAPI::TEST(WKWebView, AttributedStringWithSourceApplicationBundleID)):
(TestWebKitAPI::TEST(WKWebView, TextWithWebFontAsAttributedString)):
(TestWebKitAPI::TEST(WKWebView, AttributedStringAndCDATASection)):
(TestWebKitAPI::TEST(WKWebView, RequestAllTextRunsWithSubframes)):
(TEST(WKWebView, GetContentsShouldReturnString)): Deleted.
(TEST(WKWebView, GetContentsShouldFailWhenClosingPage)): Deleted.
(TEST(WKWebView, GetContentsOfAllFramesShouldReturnString)): Deleted.
(TEST(WKWebView, GetContentsShouldReturnAttributedString)): Deleted.
(TEST(WKWebView, GetContentsWithOpticallySizedFontShouldReturnAttributedString)): Deleted.
(TEST(WKWebView, AttributedStringAccessibilityLabel)): Deleted.
(TEST(WKWebView, AttributedStringAttributeTypes)): Deleted.
(TEST(WKWebView, AttributedStringFromTable)): Deleted.
(TEST(WKWebView, AttributedStringWithLinksInTableCell)): Deleted.
(TEST(WKWebView, AttributedStringFromList)): Deleted.
(TEST(WKWebView, AttributedStringWithoutNetworkLoads)): Deleted.
(TEST(WKWebView, AttributedStringWithSourceApplicationBundleID)): Deleted.
(TEST(WKWebView, TextWithWebFontAsAttributedString)): Deleted.
(TEST(WKWebView, AttributedStringAndCDATASection)): Deleted.
(TEST(WKWebView, AttributedStringIncludesUserSelectNoneContent)): Deleted.
* Tools/TestWebKitAPI/Tests/WebKitCocoa/nested-frames.html:
* Tools/TestWebKitAPI/Tests/WebKitCocoa/subframes.html: Added.

Add a basic API test, verifying that the new SPI method finds text inside of nested subframes and
passes back rects in the web view's coordinate space.

Canonical link: https://commits.webkit.org/285706@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications