<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[285725] trunk/Source/WebCore</title>
</head>
<body>

<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt;  }
#msg dl a { font-weight: bold}
#msg dl a:link    { color:#fc3; }
#msg dl a:active  { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff  {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/285725">285725</a></dd>
<dt>Author</dt> <dd>wenson_hsieh@apple.com</dd>
<dt>Date</dt> <dd>2021-11-12 08:52:20 -0800 (Fri, 12 Nov 2021)</dd>
</dl>

<h3>Log Message</h3>
<pre>Move subtree update logic in ImageOverlay::updateWithTextRecognitionResult() into a separate helper
https://bugs.webkit.org/show_bug.cgi?id=233010

Reviewed by Aditya Keerthi.

Split `updateWithTextRecognitionResult()` into two phases: the first of which updates the UA shadow DOM to
reflect the given text recognition results, and a second phase that updates inline styles for each of the image
overlay elements by mapping normalized OCR quads onto rotated bounding rects in client coordinates. This will
make it easier to add support for representing `TextRecognitionBlockData` as image overlay content in the next
patch.

* dom/ImageOverlay.cpp:
(WebCore::ImageOverlay::imageOverlayLineClass):
(WebCore::ImageOverlay::imageOverlayTextClass):
(WebCore::ImageOverlay::updateSubtree):

Now that this is all namespaced inside `ImageOverlay`, we can also simplify some of these names. Instead of
TextRecognitionLineElements and TextRecognitionElements, we can just call them LineElements and Elements.

(WebCore::ImageOverlay::updateWithTextRecognitionResult):</pre>

<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourceWebCoreChangeLog">trunk/Source/WebCore/ChangeLog</a></li>
<li><a href="#trunkSourceWebCoredomImageOverlaycpp">trunk/Source/WebCore/dom/ImageOverlay.cpp</a></li>
</ul>

</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourceWebCoreChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/ChangeLog (285724 => 285725)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/ChangeLog   2021-11-12 16:39:49 UTC (rev 285724)
+++ trunk/Source/WebCore/ChangeLog      2021-11-12 16:52:20 UTC (rev 285725)
</span><span class="lines">@@ -1,3 +1,26 @@
</span><ins>+2021-11-12  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        Move subtree update logic in ImageOverlay::updateWithTextRecognitionResult() into a separate helper
+        https://bugs.webkit.org/show_bug.cgi?id=233010
+
+        Reviewed by Aditya Keerthi.
+
+        Split `updateWithTextRecognitionResult()` into two phases: the first of which updates the UA shadow DOM to
+        reflect the given text recognition results, and a second phase that updates inline styles for each of the image
+        overlay elements by mapping normalized OCR quads onto rotated bounding rects in client coordinates. This will
+        make it easier to add support for representing `TextRecognitionBlockData` as image overlay content in the next
+        patch.
+
+        * dom/ImageOverlay.cpp:
+        (WebCore::ImageOverlay::imageOverlayLineClass):
+        (WebCore::ImageOverlay::imageOverlayTextClass):
+        (WebCore::ImageOverlay::updateSubtree):
+
+        Now that this is all namespaced inside `ImageOverlay`, we can also simplify some of these names. Instead of
+        TextRecognitionLineElements and TextRecognitionElements, we can just call them LineElements and Elements.
+
+        (WebCore::ImageOverlay::updateWithTextRecognitionResult):
+
</ins><span class="cx"> 2021-11-12  Adrian Perez de Castro  <aperez@igalia.com>
</span><span class="cx"> 
</span><span class="cx">         Some C++ source files use #pragma once
</span></span></pre></div>
<a id="trunkSourceWebCoredomImageOverlaycpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/dom/ImageOverlay.cpp (285724 => 285725)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/dom/ImageOverlay.cpp        2021-11-12 16:39:49 UTC (rev 285724)
+++ trunk/Source/WebCore/dom/ImageOverlay.cpp   2021-11-12 16:52:20 UTC (rev 285725)
</span><span class="lines">@@ -69,6 +69,22 @@
</span><span class="cx">     return className;
</span><span class="cx"> }
</span><span class="cx"> 
</span><ins>+#if ENABLE(IMAGE_ANALYSIS)
+
+static const AtomString& imageOverlayLineClass()
+{
+    static MainThreadNeverDestroyed<const AtomString> className("image-overlay-line", AtomString::ConstructFromLiteral);
+    return className;
+}
+
+static const AtomString& imageOverlayTextClass()
+{
+    static MainThreadNeverDestroyed<const AtomString> className("image-overlay-text", AtomString::ConstructFromLiteral);
+    return className;
+}
+
+#endif // ENABLE(IMAGE_ANALYSIS)
+
</ins><span class="cx"> bool hasOverlay(const HTMLElement& element)
</span><span class="cx"> {
</span><span class="cx">     auto shadowRoot = element.shadowRoot();
</span><span class="lines">@@ -166,24 +182,21 @@
</span><span class="cx">     return enclosingIntRect(downcast<RenderImage>(*renderer).replacedContentRect());
</span><span class="cx"> }
</span><span class="cx"> 
</span><del>-void updateWithTextRecognitionResult(HTMLElement& element, const TextRecognitionResult& result, CacheTextRecognitionResults cacheTextRecognitionResults)
-{
-    static MainThreadNeverDestroyed<const AtomString> imageOverlayLineClass("image-overlay-line", AtomString::ConstructFromLiteral);
-    static MainThreadNeverDestroyed<const AtomString> imageOverlayTextClass("image-overlay-text", AtomString::ConstructFromLiteral);
</del><ins>+struct LineElements {
+    Ref<HTMLDivElement> line;
+    Vector<Ref<HTMLElement>> children;
+};
</ins><span class="cx"> 
</span><del>-    struct TextRecognitionLineElements {
-        Ref<HTMLDivElement> line;
-        Vector<Ref<HTMLElement>> children;
-    };
</del><ins>+struct Elements {
+    RefPtr<HTMLDivElement> root;
+    Vector<LineElements> lines;
+    Vector<Ref<HTMLDivElement>> dataDetectors;
+};
</ins><span class="cx"> 
</span><del>-    struct TextRecognitionElements {
-        RefPtr<HTMLDivElement> root;
-        Vector<TextRecognitionLineElements> lines;
-        Vector<Ref<HTMLDivElement>> dataDetectors;
-    };
-
-    bool hadExistingTextRecognitionElements = false;
-    TextRecognitionElements textRecognitionElements;
</del><ins>+static Elements updateSubtree(HTMLElement& element, const TextRecognitionResult& result)
+{
+    bool hadExistingElements = false;
+    Elements elements;
</ins><span class="cx">     RefPtr<HTMLElement> mediaControlsContainer;
</span><span class="cx">     if (RefPtr shadowRoot = element.shadowRoot()) {
</span><span class="cx"> #if ENABLE(MODERN_MEDIA_CONTROLS)
</span><span class="lines">@@ -207,8 +220,8 @@
</span><span class="cx">                 containerForImageOverlay = shadowRoot;
</span><span class="cx">             for (auto& child : childrenOfType<HTMLDivElement>(*containerForImageOverlay)) {
</span><span class="cx">                 if (child.getIdAttribute() == imageOverlayElementIdentifier()) {
</span><del>-                    textRecognitionElements.root = &child;
-                    hadExistingTextRecognitionElements = true;
</del><ins>+                    elements.root = &child;
+                    hadExistingElements = true;
</ins><span class="cx">                     continue;
</span><span class="cx">                 }
</span><span class="cx">             }
</span><span class="lines">@@ -215,30 +228,30 @@
</span><span class="cx">         }
</span><span class="cx">     }
</span><span class="cx"> 
</span><del>-    if (textRecognitionElements.root) {
-        for (auto& lineOrDataDetector : childrenOfType<HTMLDivElement>(*textRecognitionElements.root)) {
</del><ins>+    if (elements.root) {
+        for (auto& lineOrDataDetector : childrenOfType<HTMLDivElement>(*elements.root)) {
</ins><span class="cx">             if (!lineOrDataDetector.hasClass())
</span><span class="cx">                 continue;
</span><span class="cx"> 
</span><del>-            if (lineOrDataDetector.classList().contains(imageOverlayLineClass)) {
-                TextRecognitionLineElements lineElements { lineOrDataDetector, { } };
</del><ins>+            if (lineOrDataDetector.classList().contains(imageOverlayLineClass())) {
+                LineElements lineElements { lineOrDataDetector, { } };
</ins><span class="cx">                 for (auto& text : childrenOfType<HTMLDivElement>(lineOrDataDetector))
</span><span class="cx">                     lineElements.children.append(text);
</span><del>-                textRecognitionElements.lines.append(WTFMove(lineElements));
</del><ins>+                elements.lines.append(WTFMove(lineElements));
</ins><span class="cx">             } else if (lineOrDataDetector.classList().contains(imageOverlayDataDetectorClassName()))
</span><del>-                textRecognitionElements.dataDetectors.append(lineOrDataDetector);
</del><ins>+                elements.dataDetectors.append(lineOrDataDetector);
</ins><span class="cx">         }
</span><span class="cx"> 
</span><del>-        bool canUseExistingTextRecognitionElements = ([&] {
-            if (result.dataDetectors.size() != textRecognitionElements.dataDetectors.size())
</del><ins>+        bool canUseExistingElements = ([&] {
+            if (result.dataDetectors.size() != elements.dataDetectors.size())
</ins><span class="cx">                 return false;
</span><span class="cx"> 
</span><del>-            if (result.lines.size() != textRecognitionElements.lines.size())
</del><ins>+            if (result.lines.size() != elements.lines.size())
</ins><span class="cx">                 return false;
</span><span class="cx"> 
</span><span class="cx">             for (size_t lineIndex = 0; lineIndex < result.lines.size(); ++lineIndex) {
</span><span class="cx">                 auto& childResults = result.lines[lineIndex].children;
</span><del>-                auto& childTextElements = textRecognitionElements.lines[lineIndex].children;
</del><ins>+                auto& childTextElements = elements.lines[lineIndex].children;
</ins><span class="cx">                 if (childResults.size() != childTextElements.size())
</span><span class="cx">                     return false;
</span><span class="cx"> 
</span><span class="lines">@@ -251,18 +264,18 @@
</span><span class="cx">             return true;
</span><span class="cx">         })();
</span><span class="cx"> 
</span><del>-        if (!canUseExistingTextRecognitionElements) {
-            textRecognitionElements.root->remove();
-            textRecognitionElements = { };
</del><ins>+        if (!canUseExistingElements) {
+            elements.root->remove();
+            elements = { };
</ins><span class="cx">         }
</span><span class="cx">     }
</span><span class="cx"> 
</span><span class="cx">     if (result.isEmpty())
</span><del>-        return;
</del><ins>+        return { };
</ins><span class="cx"> 
</span><span class="cx">     Ref document = element.document();
</span><span class="cx">     Ref shadowRoot = element.ensureUserAgentShadowRoot();
</span><del>-    if (!textRecognitionElements.root) {
</del><ins>+    if (!elements.root) {
</ins><span class="cx">         auto rootContainer = HTMLDivElement::create(document.get());
</span><span class="cx">         rootContainer->setIdAttribute(imageOverlayElementIdentifier());
</span><span class="cx">         if (document->isImageDocument())
</span><span class="lines">@@ -272,18 +285,18 @@
</span><span class="cx">             mediaControlsContainer->appendChild(rootContainer);
</span><span class="cx">         else
</span><span class="cx">             shadowRoot->appendChild(rootContainer);
</span><del>-        textRecognitionElements.root = rootContainer.copyRef();
-        textRecognitionElements.lines.reserveInitialCapacity(result.lines.size());
</del><ins>+        elements.root = rootContainer.copyRef();
+        elements.lines.reserveInitialCapacity(result.lines.size());
</ins><span class="cx">         for (auto& line : result.lines) {
</span><span class="cx">             auto lineContainer = HTMLDivElement::create(document.get());
</span><del>-            lineContainer->classList().add(imageOverlayLineClass);
</del><ins>+            lineContainer->classList().add(imageOverlayLineClass());
</ins><span class="cx">             rootContainer->appendChild(lineContainer);
</span><del>-            TextRecognitionLineElements lineElements { lineContainer, { } };
</del><ins>+            LineElements lineElements { lineContainer, { } };
</ins><span class="cx">             lineElements.children.reserveInitialCapacity(line.children.size());
</span><span class="cx">             for (size_t childIndex = 0; childIndex < line.children.size(); ++childIndex) {
</span><span class="cx">                 auto& child = line.children[childIndex];
</span><span class="cx">                 auto textContainer = HTMLDivElement::create(document.get());
</span><del>-                textContainer->classList().add(imageOverlayTextClass);
</del><ins>+                textContainer->classList().add(imageOverlayTextClass());
</ins><span class="cx">                 lineContainer->appendChild(textContainer);
</span><span class="cx">                 textContainer->appendChild(Text::create(document.get(), child.hasLeadingWhitespace ? makeString('\n', child.text) : child.text));
</span><span class="cx">                 lineElements.children.uncheckedAppend(WTFMove(textContainer));
</span><span class="lines">@@ -290,16 +303,16 @@
</span><span class="cx">             }
</span><span class="cx"> 
</span><span class="cx">             lineContainer->appendChild(HTMLBRElement::create(document.get()));
</span><del>-            textRecognitionElements.lines.uncheckedAppend(WTFMove(lineElements));
</del><ins>+            elements.lines.uncheckedAppend(WTFMove(lineElements));
</ins><span class="cx">         }
</span><span class="cx"> 
</span><span class="cx"> #if ENABLE(DATA_DETECTION)
</span><del>-        textRecognitionElements.dataDetectors.reserveInitialCapacity(result.dataDetectors.size());
</del><ins>+        elements.dataDetectors.reserveInitialCapacity(result.dataDetectors.size());
</ins><span class="cx">         for (auto& dataDetector : result.dataDetectors) {
</span><span class="cx">             auto dataDetectorContainer = DataDetection::createElementForImageOverlay(document.get(), dataDetector);
</span><span class="cx">             dataDetectorContainer->classList().add(imageOverlayDataDetectorClassName());
</span><span class="cx">             rootContainer->appendChild(dataDetectorContainer);
</span><del>-            textRecognitionElements.dataDetectors.uncheckedAppend(WTFMove(dataDetectorContainer));
</del><ins>+            elements.dataDetectors.uncheckedAppend(WTFMove(dataDetectorContainer));
</ins><span class="cx">         }
</span><span class="cx"> #endif // ENABLE(DATA_DETECTION)
</span><span class="cx"> 
</span><span class="lines">@@ -307,7 +320,7 @@
</span><span class="cx">             element.setInlineStyleProperty(CSSPropertyWebkitUserSelect, CSSValueText);
</span><span class="cx">     }
</span><span class="cx"> 
</span><del>-    if (!hadExistingTextRecognitionElements) {
</del><ins>+    if (!hadExistingElements) {
</ins><span class="cx">         static MainThreadNeverDestroyed<const String> shadowStyle(StringImpl::createWithoutCopying(imageOverlayUserAgentStyleSheet, sizeof(imageOverlayUserAgentStyleSheet)));
</span><span class="cx">         auto style = HTMLStyleElement::create(HTMLNames::styleTag, document.get(), false);
</span><span class="cx">         style->setTextContent(shadowStyle);
</span><span class="lines">@@ -314,6 +327,16 @@
</span><span class="cx">         shadowRoot->appendChild(WTFMove(style));
</span><span class="cx">     }
</span><span class="cx"> 
</span><ins>+    return elements;
+}
+
+void updateWithTextRecognitionResult(HTMLElement& element, const TextRecognitionResult& result, CacheTextRecognitionResults cacheTextRecognitionResults)
+{
+    auto elements = updateSubtree(element, result);
+    if (!elements.root)
+        return;
+
+    Ref document = element.document();
</ins><span class="cx">     document->updateLayoutIgnorePendingStylesheets();
</span><span class="cx"> 
</span><span class="cx">     auto* renderer = element.renderer();
</span><span class="lines">@@ -332,7 +355,7 @@
</span><span class="cx"> 
</span><span class="cx">     bool applyUserSelectAll = document->isImageDocument() || renderer->style().userSelect() != UserSelect::None;
</span><span class="cx">     for (size_t lineIndex = 0; lineIndex < result.lines.size(); ++lineIndex) {
</span><del>-        auto& lineElements = textRecognitionElements.lines[lineIndex];
</del><ins>+        auto& lineElements = elements.lines[lineIndex];
</ins><span class="cx">         auto& lineContainer = lineElements.line;
</span><span class="cx">         auto& line = result.lines[lineIndex];
</span><span class="cx">         auto lineQuad = convertToContainerCoordinates(line.normalizedQuad);
</span><span class="lines">@@ -423,7 +446,7 @@
</span><span class="cx"> 
</span><span class="cx"> #if ENABLE(DATA_DETECTION)
</span><span class="cx">     for (size_t index = 0; index < result.dataDetectors.size(); ++index) {
</span><del>-        auto dataDetectorContainer = textRecognitionElements.dataDetectors[index];
</del><ins>+        auto dataDetectorContainer = elements.dataDetectors[index];
</ins><span class="cx">         auto& dataDetector = result.dataDetectors[index];
</span><span class="cx">         if (dataDetector.normalizedQuads.isEmpty())
</span><span class="cx">             continue;
</span></span></pre>
</div>
</div>

</body>
</html>