<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[279349] trunk</title>
</head>
<body>

<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt;  }
#msg dl a { font-weight: bold}
#msg dl a:link    { color:#fc3; }
#msg dl a:active  { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff  {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/279349">279349</a></dd>
<dt>Author</dt> <dd>wenson_hsieh@apple.com</dd>
<dt>Date</dt> <dd>2021-06-28 14:44:51 -0700 (Mon, 28 Jun 2021)</dd>
</dl>

<h3>Log Message</h3>
<pre>Live Text selections inside images is misaligned when "object-fit" is not "fill"
https://bugs.webkit.org/show_bug.cgi?id=227453

Reviewed by Tim Horton.

Source/WebCore:

We currently use the bounds of the image overlay host element when injecting transformed Live Text into image
element UA shadow roots. However, this causes text to lay out in the wrong place relative to the image when
using any "object-fit" values that are not "fill". To address this, use the `replacedContentRect()` of the image
renderer when injecting OCR text quads instead.

Tests: fast/images/text-recognition/image-overlay-object-fit-change.html
       fast/images/text-recognition/image-overlay-object-fit.html

* html/HTMLElement.cpp:
(WebCore::HTMLElement::updateWithTextRecognitionResult): See description above for more details.
* page/Page.cpp:
(WebCore::Page::updateElementsWithTextRecognitionResults):

We also refactor this code so that we don't immediately try to update text recognition results, and instead
queue an internal async task to do so. Immediately performing the update here can potentially cause a debug
assertion due to leaving us in a state where we require layout immediately after a rendering update.

(WebCore::Page::cacheTextRecognitionResult):

Additionally adjust the text recognition result caching mechanism to update image overlay content when changing
the replaced content rect, rather than the element's offset width or height. This ensures that changing CSS
"object-fit" values dynamically for images with Live Text causes Live Text bounds to stay up to date.

* page/Page.h:

LayoutTests:

Add a couple of new tests, and do some minor cleanup in an existing test.

* fast/images/text-recognition/image-overlay-object-fit-change-expected.txt: Added.
* fast/images/text-recognition/image-overlay-object-fit-change.html: Added.

Add a layout test to verify that dynamically changing CSS "object-fit" values for an image with Live Text causes
the dimensions of the Live Text to update as well.

* fast/images/text-recognition/image-overlay-object-fit-expected.txt: Added.
* fast/images/text-recognition/image-overlay-object-fit.html: Added.

Add a layout test to verify that Live Text is injected correctly when using all 5 values of "object-fit".

* fast/images/text-recognition/image-overlay-size-change.html:

Clean up this layout test a bit: remove an unnecessarily included script file, add a missing `head` tag and
don't try to inject an image overlay 10 times.</pre>

<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkLayoutTestsChangeLog">trunk/LayoutTests/ChangeLog</a></li>
<li><a href="#trunkLayoutTestsfastimagestextrecognitionimageoverlaysizechangehtml">trunk/LayoutTests/fast/images/text-recognition/image-overlay-size-change.html</a></li>
<li><a href="#trunkSourceWebCoreChangeLog">trunk/Source/WebCore/ChangeLog</a></li>
<li><a href="#trunkSourceWebCorehtmlHTMLElementcpp">trunk/Source/WebCore/html/HTMLElement.cpp</a></li>
<li><a href="#trunkSourceWebCorepagePagecpp">trunk/Source/WebCore/page/Page.cpp</a></li>
<li><a href="#trunkSourceWebCorepagePageh">trunk/Source/WebCore/page/Page.h</a></li>
</ul>

<h3>Added Paths</h3>
<ul>
<li><a href="#trunkLayoutTestsfastimagestextrecognitionimageoverlayobjectfitchangeexpectedtxt">trunk/LayoutTests/fast/images/text-recognition/image-overlay-object-fit-change-expected.txt</a></li>
<li><a href="#trunkLayoutTestsfastimagestextrecognitionimageoverlayobjectfitchangehtml">trunk/LayoutTests/fast/images/text-recognition/image-overlay-object-fit-change.html</a></li>
<li><a href="#trunkLayoutTestsfastimagestextrecognitionimageoverlayobjectfitexpectedtxt">trunk/LayoutTests/fast/images/text-recognition/image-overlay-object-fit-expected.txt</a></li>
<li><a href="#trunkLayoutTestsfastimagestextrecognitionimageoverlayobjectfithtml">trunk/LayoutTests/fast/images/text-recognition/image-overlay-object-fit.html</a></li>
</ul>

</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkLayoutTestsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/LayoutTests/ChangeLog (279348 => 279349)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/ChangeLog      2021-06-28 21:24:48 UTC (rev 279348)
+++ trunk/LayoutTests/ChangeLog 2021-06-28 21:44:51 UTC (rev 279349)
</span><span class="lines">@@ -1,3 +1,28 @@
</span><ins>+2021-06-28  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        Live Text selections inside images is misaligned when "object-fit" is not "fill"
+        https://bugs.webkit.org/show_bug.cgi?id=227453
+
+        Reviewed by Tim Horton.
+
+        Add a couple of new tests, and do some minor cleanup in an existing test.
+
+        * fast/images/text-recognition/image-overlay-object-fit-change-expected.txt: Added.
+        * fast/images/text-recognition/image-overlay-object-fit-change.html: Added.
+
+        Add a layout test to verify that dynamically changing CSS "object-fit" values for an image with Live Text causes
+        the dimensions of the Live Text to update as well.
+
+        * fast/images/text-recognition/image-overlay-object-fit-expected.txt: Added.
+        * fast/images/text-recognition/image-overlay-object-fit.html: Added.
+
+        Add a layout test to verify that Live Text is injected correctly when using all 5 values of "object-fit".
+
+        * fast/images/text-recognition/image-overlay-size-change.html:
+
+        Clean up this layout test a bit: remove an unnecessarily included script file, add a missing `head` tag and
+        don't try to inject an image overlay 10 times.
+
</ins><span class="cx"> 2021-06-28  Amir Mark Jr  <amir_mark@apple.com>
</span><span class="cx"> 
</span><span class="cx">         [Catalina WK2 Debug/ iOS 14 Debug] fast/css-custom-paint/out-of-memory-while-adding-worklet-module.html is a flaky timeout
</span></span></pre></div>
<a id="trunkLayoutTestsfastimagestextrecognitionimageoverlayobjectfitchangeexpectedtxt"></a>
<div class="addfile"><h4>Added: trunk/LayoutTests/fast/images/text-recognition/image-overlay-object-fit-change-expected.txt (0 => 279349)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/fast/images/text-recognition/image-overlay-object-fit-change-expected.txt                              (rev 0)
+++ trunk/LayoutTests/fast/images/text-recognition/image-overlay-object-fit-change-expected.txt 2021-06-28 21:44:51 UTC (rev 279349)
</span><span class="lines">@@ -0,0 +1,8 @@
</span><ins>+PASS width is 50
+PASS height is 25
+PASS width became 100
+PASS height is 50
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
</ins></span></pre></div>
<a id="trunkLayoutTestsfastimagestextrecognitionimageoverlayobjectfitchangehtml"></a>
<div class="addfile"><h4>Added: trunk/LayoutTests/fast/images/text-recognition/image-overlay-object-fit-change.html (0 => 279349)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/fast/images/text-recognition/image-overlay-object-fit-change.html                              (rev 0)
+++ trunk/LayoutTests/fast/images/text-recognition/image-overlay-object-fit-change.html 2021-06-28 21:44:51 UTC (rev 279349)
</span><span class="lines">@@ -0,0 +1,61 @@
</span><ins>+<!DOCTYPE html>
+<html>
+<head>
+<script src="../../../resources/js-test.js"></script>
+<style>
+img {
+    width: 100px;
+    height: 200px;
+}
+</style>
+</head>
+<body>
+<img src="../resources/green-400x400.png"></img>
+<script>
+jsTestIsAsync = true;
+let image = document.querySelector("img");
+width = 0;
+height = 0;
+
+function updateImageOverlayTextDimensions() {
+    let textBoundingRect = internals.shadowRoot(image).querySelector(".image-overlay-text").getBoundingClientRect();
+    width = textBoundingRect.width;
+    height = textBoundingRect.height;
+}
+
+addEventListener("load", async () => {
+    image.style.objectFit = "contain";
+    internals.installImageOverlay(image, [
+        {
+            topLeft : new DOMPointReadOnly(0, 0),
+            topRight : new DOMPointReadOnly(0.5, 0),
+            bottomRight : new DOMPointReadOnly(0.5, 0.25),
+            bottomLeft : new DOMPointReadOnly(0, 0.25),
+            children: [
+                {
+                    text : "foo",
+                    topLeft : new DOMPointReadOnly(0, 0),
+                    topRight : new DOMPointReadOnly(0.5, 0),
+                    bottomRight : new DOMPointReadOnly(0.5, 0.25),
+                    bottomLeft : new DOMPointReadOnly(0, 0.25),
+                }
+            ]
+        }
+    ]);
+
+    updateImageOverlayTextDimensions();
+
+    shouldBe("width", "50");
+    shouldBe("height", "25");
+
+    image.style.objectFit = "cover";
+    setInterval(updateImageOverlayTextDimensions, 10);
+
+    await shouldBecomeEqual("width", "100");
+    shouldBe("height", "50");
+
+    finishJSTest();
+});
+</script>
+</body>
+</html>
</ins><span class="cx">\ No newline at end of file
</span></span></pre></div>
<a id="trunkLayoutTestsfastimagestextrecognitionimageoverlayobjectfitexpectedtxt"></a>
<div class="addfile"><h4>Added: trunk/LayoutTests/fast/images/text-recognition/image-overlay-object-fit-expected.txt (0 => 279349)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/fast/images/text-recognition/image-overlay-object-fit-expected.txt                             (rev 0)
+++ trunk/LayoutTests/fast/images/text-recognition/image-overlay-object-fit-expected.txt        2021-06-28 21:44:51 UTC (rev 279349)
</span><span class="lines">@@ -0,0 +1,10 @@
</span><ins>+
+PASS textRects['fill'] is [0, 0, 100, 25]
+PASS textRects['contain'] is [50, 0, 50, 25]
+PASS textRects['cover'] is [0, -50, 100, 50]
+PASS textRects['none'] is [-100, -150, 200, 100]
+PASS textRects['scale-down'] is [50, 0, 50, 25]
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
</ins></span></pre></div>
<a id="trunkLayoutTestsfastimagestextrecognitionimageoverlayobjectfithtml"></a>
<div class="addfile"><h4>Added: trunk/LayoutTests/fast/images/text-recognition/image-overlay-object-fit.html (0 => 279349)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/fast/images/text-recognition/image-overlay-object-fit.html                             (rev 0)
+++ trunk/LayoutTests/fast/images/text-recognition/image-overlay-object-fit.html        2021-06-28 21:44:51 UTC (rev 279349)
</span><span class="lines">@@ -0,0 +1,64 @@
</span><ins>+<!DOCTYPE html>
+<html>
+<head>
+<script src="../../../resources/js-test.js"></script>
+<style>
+body, html {
+    margin: 0;
+}
+img {
+    width: 200px;
+    height: 100px;
+    position: absolute;
+    top: 0;
+    left: 0;
+}
+</style>
+</head>
+<body>
+<img style="object-fit: fill;" src="../resources/green-400x400.png"></img>
+<img style="object-fit: contain;" src="../resources/green-400x400.png"></img>
+<img style="object-fit: cover;" src="../resources/green-400x400.png"></img>
+<img style="object-fit: none;" src="../resources/green-400x400.png"></img>
+<img style="object-fit: scale-down;" src="../resources/green-400x400.png"></img>
+<pre id="console"></pre>
+<script>
+function imageOverlayTextRect(image) {
+    let bounds = internals.shadowRoot(image).querySelector(".image-overlay-text").getBoundingClientRect();
+    return [ Math.round(bounds.left), Math.round(bounds.top), Math.round(bounds.width), Math.round(bounds.height) ];
+}
+
+addEventListener("load", async () => {
+    textRects = { };
+
+    document.querySelectorAll("img").forEach(image => {
+        internals.installImageOverlay(image, [
+            {
+                topLeft : new DOMPointReadOnly(0, 0),
+                topRight : new DOMPointReadOnly(0.5, 0),
+                bottomRight : new DOMPointReadOnly(0.5, 0.25),
+                bottomLeft : new DOMPointReadOnly(0, 0.25),
+                children: [
+                    {
+                        text : "foo",
+                        topLeft : new DOMPointReadOnly(0, 0),
+                        topRight : new DOMPointReadOnly(0.5, 0),
+                        bottomRight : new DOMPointReadOnly(0.5, 0.25),
+                        bottomLeft : new DOMPointReadOnly(0, 0.25),
+                    }
+                ]
+            }
+        ]);
+
+        textRects[getComputedStyle(image).objectFit] = imageOverlayTextRect(image);
+    });
+
+    shouldBe("textRects['fill']", "[0, 0, 100, 25]");
+    shouldBe("textRects['contain']", "[50, 0, 50, 25]");
+    shouldBe("textRects['cover']", "[0, -50, 100, 50]");
+    shouldBe("textRects['none']", "[-100, -150, 200, 100]");
+    shouldBe("textRects['scale-down']", "[50, 0, 50, 25]");
+});
+</script>
+</body>
+</html>
</ins><span class="cx">\ No newline at end of file
</span></span></pre></div>
<a id="trunkLayoutTestsfastimagestextrecognitionimageoverlaysizechangehtml"></a>
<div class="modfile"><h4>Modified: trunk/LayoutTests/fast/images/text-recognition/image-overlay-size-change.html (279348 => 279349)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/fast/images/text-recognition/image-overlay-size-change.html    2021-06-28 21:24:48 UTC (rev 279348)
+++ trunk/LayoutTests/fast/images/text-recognition/image-overlay-size-change.html       2021-06-28 21:44:51 UTC (rev 279349)
</span><span class="lines">@@ -1,7 +1,8 @@
</span><span class="cx"> <!DOCTYPE html>
</span><span class="cx"> <html>
</span><ins>+<head>
</ins><span class="cx"> <script src="../../../resources/js-test.js"></script>
</span><del>-<script src="../../../resources/ui-helper.js"></script>
</del><ins>+</head>
</ins><span class="cx"> <body>
</span><span class="cx"> <img src="../resources/green-400x400.png"></img>
</span><span class="cx"> <script>
</span><span class="lines">@@ -17,25 +18,23 @@
</span><span class="cx"> }
</span><span class="cx"> 
</span><span class="cx"> addEventListener("load", async () => {
</span><del>-    for (let i = 0; i < 10; ++i) {
-        internals.installImageOverlay(image, [
-            {
-                topLeft : new DOMPointReadOnly(0.5, 0.5),
-                topRight : new DOMPointReadOnly(1, 0.5),
-                bottomRight : new DOMPointReadOnly(1, 1),
-                bottomLeft : new DOMPointReadOnly(0.5, 1),
-                children: [
-                    {
-                        text : "Hello",
-                        topLeft : new DOMPointReadOnly(0.5, 0.5),
-                        topRight : new DOMPointReadOnly(1, 0.5),
-                        bottomRight : new DOMPointReadOnly(1, 1),
-                        bottomLeft : new DOMPointReadOnly(0.5, 1),
-                    }
-                ],
-            }
-        ]);
-    }
</del><ins>+    internals.installImageOverlay(image, [
+        {
+            topLeft : new DOMPointReadOnly(0.5, 0.5),
+            topRight : new DOMPointReadOnly(1, 0.5),
+            bottomRight : new DOMPointReadOnly(1, 1),
+            bottomLeft : new DOMPointReadOnly(0.5, 1),
+            children: [
+                {
+                    text : "Hello",
+                    topLeft : new DOMPointReadOnly(0.5, 0.5),
+                    topRight : new DOMPointReadOnly(1, 0.5),
+                    bottomRight : new DOMPointReadOnly(1, 1),
+                    bottomLeft : new DOMPointReadOnly(0.5, 1),
+                }
+            ],
+        }
+    ]);
</ins><span class="cx"> 
</span><span class="cx">     updateImageOverlayTextDimensions();
</span><span class="cx"> 
</span></span></pre></div>
<a id="trunkSourceWebCoreChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/ChangeLog (279348 => 279349)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/ChangeLog   2021-06-28 21:24:48 UTC (rev 279348)
+++ trunk/Source/WebCore/ChangeLog      2021-06-28 21:44:51 UTC (rev 279349)
</span><span class="lines">@@ -1,3 +1,35 @@
</span><ins>+2021-06-28  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        Live Text selections inside images is misaligned when "object-fit" is not "fill"
+        https://bugs.webkit.org/show_bug.cgi?id=227453
+
+        Reviewed by Tim Horton.
+
+        We currently use the bounds of the image overlay host element when injecting transformed Live Text into image
+        element UA shadow roots. However, this causes text to lay out in the wrong place relative to the image when
+        using any "object-fit" values that are not "fill". To address this, use the `replacedContentRect()` of the image
+        renderer when injecting OCR text quads instead.
+
+        Tests: fast/images/text-recognition/image-overlay-object-fit-change.html
+               fast/images/text-recognition/image-overlay-object-fit.html
+
+        * html/HTMLElement.cpp:
+        (WebCore::HTMLElement::updateWithTextRecognitionResult): See description above for more details.
+        * page/Page.cpp:
+        (WebCore::Page::updateElementsWithTextRecognitionResults):
+
+        We also refactor this code so that we don't immediately try to update text recognition results, and instead
+        queue an internal async task to do so. Immediately performing the update here can potentially cause a debug
+        assertion due to leaving us in a state where we require layout immediately after a rendering update.
+
+        (WebCore::Page::cacheTextRecognitionResult):
+
+        Additionally adjust the text recognition result caching mechanism to update image overlay content when changing
+        the replaced content rect, rather than the element's offset width or height. This ensures that changing CSS
+        "object-fit" values dynamically for images with Live Text causes Live Text bounds to stay up to date.
+
+        * page/Page.h:
+
</ins><span class="cx"> 2021-06-28  Sam Weinig  <weinig@apple.com>
</span><span class="cx"> 
</span><span class="cx">         [Modern Media Controls] Support customizing the media controls via WebKitAdditions
</span></span></pre></div>
<a id="trunkSourceWebCorehtmlHTMLElementcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/html/HTMLElement.cpp (279348 => 279349)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/html/HTMLElement.cpp        2021-06-28 21:24:48 UTC (rev 279348)
+++ trunk/Source/WebCore/html/HTMLElement.cpp   2021-06-28 21:44:51 UTC (rev 279349)
</span><span class="lines">@@ -1456,13 +1456,6 @@
</span><span class="cx">             setInlineStyleProperty(CSSPropertyWebkitUserSelect, CSSValueText);
</span><span class="cx">     }
</span><span class="cx"> 
</span><del>-    if (auto* renderer = this->renderer()) {
-        if (!is<RenderImage>(renderer))
-            return;
-
-        downcast<RenderImage>(*renderer).setHasImageOverlay();
-    }
-
</del><span class="cx">     if (!hadExistingTextRecognitionElements) {
</span><span class="cx">         static MainThreadNeverDestroyed<const String> shadowStyle(StringImpl::createWithoutCopying(imageOverlayUserAgentStyleSheet, sizeof(imageOverlayUserAgentStyleSheet)));
</span><span class="cx">         auto style = HTMLStyleElement::create(HTMLNames::styleTag, document(), false);
</span><span class="lines">@@ -1470,17 +1463,30 @@
</span><span class="cx">         shadowRoot->appendChild(WTFMove(style));
</span><span class="cx">     }
</span><span class="cx"> 
</span><del>-    IntSize containerSize { offsetWidth(), offsetHeight() };
</del><ins>+    document().updateLayoutIgnorePendingStylesheets();
+
+    auto* renderer = this->renderer();
+    if (!is<RenderImage>(renderer))
+        return;
+
+    downcast<RenderImage>(*renderer).setHasImageOverlay();
+
+    auto containerRect = enclosingIntRect(downcast<RenderImage>(*renderer).replacedContentRect());
+    auto convertToContainerCoordinates = [&](const FloatQuad& normalizedQuad) {
+        auto quad = normalizedQuad;
+        quad.scale(containerRect.width(), containerRect.height());
+        quad.move(containerRect.x(), containerRect.y());
+        return quad;
+    };
+
</ins><span class="cx">     for (size_t lineIndex = 0; lineIndex < result.lines.size(); ++lineIndex) {
</span><span class="cx">         auto& lineElements = textRecognitionElements.lines[lineIndex];
</span><span class="cx">         auto& lineContainer = lineElements.line;
</span><span class="cx">         auto& line = result.lines[lineIndex];
</span><del>-        auto lineQuad = line.normalizedQuad;
</del><ins>+        auto lineQuad = convertToContainerCoordinates(line.normalizedQuad);
</ins><span class="cx">         if (lineQuad.isEmpty())
</span><span class="cx">             continue;
</span><span class="cx"> 
</span><del>-        lineQuad.scale(containerSize.width(), containerSize.height());
-
</del><span class="cx">         auto lineBounds = rotatedBoundingRectWithMinimumAngleOfRotation(lineQuad, 0.01);
</span><span class="cx">         lineContainer->setInlineStyleProperty(CSSPropertyWidth, lineBounds.size.width(), CSSUnitType::CSS_PX);
</span><span class="cx">         lineContainer->setInlineStyleProperty(CSSPropertyHeight, lineBounds.size.height(), CSSUnitType::CSS_PX);
</span><span class="lines">@@ -1500,8 +1506,7 @@
</span><span class="cx">         };
</span><span class="cx"> 
</span><span class="cx">         auto offsetsAlongHorizontalAxis = line.children.map([&](auto& child) -> WTF::Range<float> {
</span><del>-            auto textQuad = child.normalizedQuad;
-            textQuad.scale(containerSize.width(), containerSize.height());
</del><ins>+            auto textQuad = convertToContainerCoordinates(child.normalizedQuad);
</ins><span class="cx">             return {
</span><span class="cx">                 offsetAlongHorizontalAxis(textQuad.p1(), textQuad.p4()),
</span><span class="cx">                 offsetAlongHorizontalAxis(textQuad.p2(), textQuad.p3())
</span><span class="lines">@@ -1575,9 +1580,7 @@
</span><span class="cx">             continue;
</span><span class="cx"> 
</span><span class="cx">         // FIXME: We should come up with a way to coalesce the bounding quads into one or more rotated rects with the same angle of rotation.
</span><del>-        auto targetQuad = dataDetector.normalizedQuads.first();
-        targetQuad.scale(containerSize.width(), containerSize.height());
-
</del><ins>+        auto targetQuad = convertToContainerCoordinates(dataDetector.normalizedQuads.first());
</ins><span class="cx">         auto targetBounds = rotatedBoundingRectWithMinimumAngleOfRotation(targetQuad, 0.01);
</span><span class="cx">         dataDetectorContainer->setInlineStyleProperty(CSSPropertyWidth, targetBounds.size.width(), CSSUnitType::CSS_PX);
</span><span class="cx">         dataDetectorContainer->setInlineStyleProperty(CSSPropertyHeight, targetBounds.size.height(), CSSUnitType::CSS_PX);
</span><span class="lines">@@ -1595,7 +1598,7 @@
</span><span class="cx"> 
</span><span class="cx">     if (cacheTextRecognitionResults == CacheTextRecognitionResults::Yes) {
</span><span class="cx">         if (auto* page = document().page())
</span><del>-            page->cacheTextRecognitionResult(*this, containerSize, result);
</del><ins>+            page->cacheTextRecognitionResult(*this, containerRect, result);
</ins><span class="cx">     }
</span><span class="cx"> }
</span><span class="cx"> 
</span></span></pre></div>
<a id="trunkSourceWebCorepagePagecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/page/Page.cpp (279348 => 279349)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/page/Page.cpp       2021-06-28 21:24:48 UTC (rev 279348)
+++ trunk/Source/WebCore/page/Page.cpp  2021-06-28 21:44:51 UTC (rev 279349)
</span><span class="lines">@@ -108,6 +108,7 @@
</span><span class="cx"> #include "ProgressTracker.h"
</span><span class="cx"> #include "Range.h"
</span><span class="cx"> #include "RenderDescendantIterator.h"
</span><ins>+#include "RenderImage.h"
</ins><span class="cx"> #include "RenderLayerCompositor.h"
</span><span class="cx"> #include "RenderTheme.h"
</span><span class="cx"> #include "RenderView.h"
</span><span class="lines">@@ -3618,19 +3619,34 @@
</span><span class="cx">         return !elementAndResult.first;
</span><span class="cx">     });
</span><span class="cx"> 
</span><del>-    for (auto& [element, resultAndSize] : m_textRecognitionResultsByElement) {
-        if (!element || !element->isConnected())
</del><ins>+    Vector<std::pair<Ref<HTMLElement>, TextRecognitionResult>> elementsToUpdate;
+    for (auto& [element, resultAndRect] : m_textRecognitionResultsByElement) {
+        if (!element->isConnected())
</ins><span class="cx">             continue;
</span><span class="cx"> 
</span><ins>+        auto& [result, containerRect] = resultAndRect;
</ins><span class="cx">         auto protectedElement = makeRef(*element);
</span><del>-        auto& [result, offsetSize] = resultAndSize;
-        IntSize newOffsetSize { protectedElement->offsetWidth(), protectedElement->offsetHeight() };
-        if (offsetSize == newOffsetSize)
</del><ins>+        auto renderer = protectedElement->renderer();
+        if (!is<RenderImage>(renderer))
</ins><span class="cx">             continue;
</span><span class="cx"> 
</span><del>-        offsetSize = newOffsetSize;
-        protectedElement->updateWithTextRecognitionResult(result, HTMLElement::CacheTextRecognitionResults::No);
</del><ins>+        auto newContainerRect = enclosingIntRect(downcast<RenderImage>(*renderer).replacedContentRect());
+        if (containerRect == newContainerRect)
+            continue;
+
+        containerRect = newContainerRect;
+        elementsToUpdate.append({ WTFMove(protectedElement), result });
</ins><span class="cx">     }
</span><ins>+
+    for (auto& [element, result] : elementsToUpdate) {
+        element->document().eventLoop().queueTask(TaskSource::InternalAsyncTask, [result = TextRecognitionResult { result }, weakElement = makeWeakPtr(element.get())] {
+            auto element = makeRefPtr(weakElement.get());
+            if (!element)
+                return;
+
+            element->updateWithTextRecognitionResult(result, HTMLElement::CacheTextRecognitionResults::No);
+        });
+    }
</ins><span class="cx"> }
</span><span class="cx"> 
</span><span class="cx"> bool Page::hasCachedTextRecognitionResult(const HTMLElement& element) const
</span><span class="lines">@@ -3638,7 +3654,7 @@
</span><span class="cx">     return m_elementsWithTextRecognitionResults.contains(element);
</span><span class="cx"> }
</span><span class="cx"> 
</span><del>-void Page::cacheTextRecognitionResult(const HTMLElement& element, const IntSize& offsetSize, const TextRecognitionResult& result)
</del><ins>+void Page::cacheTextRecognitionResult(const HTMLElement& element, const IntRect& containerRect, const TextRecognitionResult& result)
</ins><span class="cx"> {
</span><span class="cx">     m_elementsWithTextRecognitionResults.add(element);
</span><span class="cx"> 
</span><span class="lines">@@ -3647,9 +3663,9 @@
</span><span class="cx">     });
</span><span class="cx"> 
</span><span class="cx">     if (index == notFound)
</span><del>-        m_textRecognitionResultsByElement.append({ makeWeakPtr(element), { result, offsetSize } });
</del><ins>+        m_textRecognitionResultsByElement.append({ makeWeakPtr(element), { result, containerRect } });
</ins><span class="cx">     else
</span><del>-        m_textRecognitionResultsByElement[index].second = { result, offsetSize };
</del><ins>+        m_textRecognitionResultsByElement[index].second = { result, containerRect };
</ins><span class="cx"> 
</span><span class="cx">     m_textRecognitionResultsByElement.removeAllMatching([](auto& elementAndResult) {
</span><span class="cx">         return !elementAndResult.first;
</span></span></pre></div>
<a id="trunkSourceWebCorepagePageh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/page/Page.h (279348 => 279349)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/page/Page.h 2021-06-28 21:24:48 UTC (rev 279348)
+++ trunk/Source/WebCore/page/Page.h    2021-06-28 21:44:51 UTC (rev 279349)
</span><span class="lines">@@ -863,7 +863,7 @@
</span><span class="cx"> 
</span><span class="cx"> #if ENABLE(IMAGE_ANALYSIS)
</span><span class="cx">     WEBCORE_EXPORT bool hasCachedTextRecognitionResult(const HTMLElement&) const;
</span><del>-    void cacheTextRecognitionResult(const HTMLElement&, const IntSize& offsetSize, const TextRecognitionResult&);
</del><ins>+    void cacheTextRecognitionResult(const HTMLElement&, const IntRect& containerRect, const TextRecognitionResult&);
</ins><span class="cx"> #endif
</span><span class="cx"> 
</span><span class="cx"> private:
</span><span class="lines">@@ -1189,7 +1189,7 @@
</span><span class="cx"> 
</span><span class="cx"> #if ENABLE(IMAGE_ANALYSIS)
</span><span class="cx">     // FIXME: These should be refactored to use a weak hash map of HTMLElement to std::pair<TextRecognitionResult, IntSize>.
</span><del>-    Vector<std::pair<WeakPtr<HTMLElement>, std::pair<TextRecognitionResult, IntSize>>> m_textRecognitionResultsByElement;
</del><ins>+    Vector<std::pair<WeakPtr<HTMLElement>, std::pair<TextRecognitionResult, IntRect>>> m_textRecognitionResultsByElement;
</ins><span class="cx">     WeakHashSet<HTMLElement> m_elementsWithTextRecognitionResults;
</span><span class="cx"> #endif
</span><span class="cx"> };
</span></span></pre>
</div>
</div>

</body>
</html>