[webkit-changes] [WebKit/WebKit] 503f7e: Add additional WKWebView SPI to extract rendered t...

[Wenson Hsieh]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 503f7e1ff8cef2d718e4dfa3612e1d9907d81e67
      https://github.com/WebKit/WebKit/commit/503f7e1ff8cef2d718e4dfa3612e1d9907d81e67
  Author: Wenson Hsieh <wenson_hsieh at apple.com>
  Date:   2024-03-05 (Tue, 05 Mar 2024)

  Changed paths:
    A LayoutTests/fast/text-extraction/extract-rendered-text-with-subframe-expected.txt
    A LayoutTests/fast/text-extraction/extract-rendered-text-with-subframe.html
    A LayoutTests/fast/text-extraction/resources/subframe.html
    M LayoutTests/resources/ui-helper.js
    M Source/WebCore/page/text-extraction/TextExtraction.cpp
    M Source/WebCore/page/text-extraction/TextExtraction.h
    M Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm
    M Source/WebKit/UIProcess/API/Cocoa/WKWebViewPrivate.h
    M Source/WebKit/UIProcess/WebAuthentication/AuthenticatorManager.cpp
    M Source/WebKit/UIProcess/WebPageProxy.cpp
    M Source/WebKit/UIProcess/WebPageProxy.h
    M Source/WebKit/WebProcess/WebPage/WebPage.cpp
    M Source/WebKit/WebProcess/WebPage/WebPage.h
    M Source/WebKit/WebProcess/WebPage/WebPage.messages.in
    M Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl
    M Tools/TestRunnerShared/UIScriptContext/UIScriptController.h
    M Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.h
    M Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.mm

  Log Message:
  -----------
  Add additional WKWebView SPI to extract rendered text for a given CSS selector in the main frame
https://bugs.webkit.org/show_bug.cgi?id=270544

Reviewed by Abrar Rahman Protyasha and Aditya Keerthi.

Add another `WKWebView` SPI method to extract rendered text from an element in the main frame, given
a CSS selector. This flattens all text content in the element subtree (including subframes) into a
single string, sorted in block layout direction order by the root view bounds of each rendered text
node.

Test: fast/text-extraction/extract-rendered-text-with-subframe.html

* LayoutTests/fast/text-extraction/extract-rendered-text-with-subframe-expected.txt: Added.
* LayoutTests/fast/text-extraction/extract-rendered-text-with-subframe.html: Added.

Add a layout test to exercise this SPI.

* LayoutTests/fast/text-extraction/resources/subframe.html: Added.
* LayoutTests/resources/ui-helper.js:
(window.UIHelper.requestTextExtraction):
(window.UIHelper.requestRenderedTextForSelector):

Add a testing hook to exercise the new SPI from layout tests.

(window.UIHelper):
* Source/WebCore/page/text-extraction/TextExtraction.cpp:
(WebCore::TextExtraction::extractRenderedText):

Implement the main logic here, by recursively traversing through the render tree to accumulate text
in a `Vector` of offsets (based on block writing direction — e.g. y-offsets, in top-down LTR text)
and strings, filtered to only contain letters (including non-Latin script) and numbers. We also
represent replaced renderers with `{w,h}`, representing the size of the renderer in root view
coordinates.

The final output is a space-separated string of extracted tokens.

* Source/WebCore/page/text-extraction/TextExtraction.h:
* Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm:
(-[WKWebView _requestRenderedTextForElementSelector:completionHandler:]):
* Source/WebKit/UIProcess/API/Cocoa/WKWebViewPrivate.h:
* Source/WebKit/UIProcess/WebAuthentication/AuthenticatorManager.cpp:
* Source/WebKit/UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::requestTextExtraction):
(WebKit::WebPageProxy::requestRenderedTextForElementSelector):
* Source/WebKit/UIProcess/WebPageProxy.h:
* Source/WebKit/WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::requestRenderedTextForElementSelector):
* Source/WebKit/WebProcess/WebPage/WebPage.h:
* Source/WebKit/WebProcess/WebPage/WebPage.messages.in:
* Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl:
* Tools/TestRunnerShared/UIScriptContext/UIScriptController.h:
(WTR::UIScriptController::requestRenderedTextForSelector):
* Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.h:
* Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.mm:
(WTR::UIScriptControllerCocoa::requestRenderedTextForSelector):

Canonical link: https://commits.webkit.org/275728@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications