<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[285609] trunk</title>
</head>
<body>

<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt;  }
#msg dl a { font-weight: bold}
#msg dl a:link    { color:#fc3; }
#msg dl a:active  { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff  {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/285609">285609</a></dd>
<dt>Author</dt> <dd>wenson_hsieh@apple.com</dd>
<dt>Date</dt> <dd>2021-11-10 16:03:06 -0800 (Wed, 10 Nov 2021)</dd>
</dl>

<h3>Log Message</h3>
<pre>Refactor some image overlay logic to work with built-in media controls
https://bugs.webkit.org/show_bug.cgi?id=232899
rdar://83173597

Reviewed by Antoine Quint and Tim Horton.

Source/WebCore:

Make various minor adjustments to allow built-in modern media controls to play well with image overlay content.
See below for more details.

* Modules/mediacontrols/MediaControlsHost.cpp:
(WebCore::MediaControlsHost::mediaControlsContainerClassName):
* Modules/mediacontrols/MediaControlsHost.h:

Add a helper function to grab the "media-controls-container" class name, which is used for the `div` element
containing built-in modern media controls. This is used below to identify existing media control container
elements when determining where to inject the image overlay root container.

* Modules/mediacontrols/MediaControlsHost.idl:
* Modules/modern-media-controls/controls/controls-bar.css:
(.controls-bar):

Z-order the media controls bar (which contains all interactible media control elements) above any image overlay
content that may coexist in the same shadow root.

* Modules/modern-media-controls/controls/media-controls.css:
(:host):

Remove `-webkit-user-select: none;` here. This was added to prevent the iOS magnifier UI from showing up when
long pressing inside a video element; we can achieve the same effect without applying this property over the
entire host element by instead changing selection logic in WebKit2.

(.media-controls):

Push the `-webkit-user-select: none;` property down into the media control children, instead of on
`.media-controls` itself.

(.media-controls > *):
* Modules/modern-media-controls/media/media-controller.js:
(MediaController):

Change this to ask for `mediaControlsContainerClassName` from the host, instead of hard-coding it to
"media-controls-container". From code inspection, there does not seem to be any codepath that passes in an
undefined `host`, except for the modern media controls layout tests (which still pass after this adjustment).

* html/HTMLElement.cpp:
(WebCore::HTMLElement::isImageOverlayText):
(WebCore::HTMLElement::removeImageOverlaySoonIfNeeded):

Adjust these helper methods to work in the case where the image overlay container is hosted underneath the media
controls container.

(WebCore::HTMLElement::updateWithTextRecognitionResult):
* html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::seekWithTolerance):
(WebCore::HTMLMediaElement::playInternal):

If needed, remove the image overlay when seeking or playing media.

Source/WebKit:

See WebCore/ChangeLog for more details.

* Shared/ios/InteractionInformationAtPosition.h:
* UIProcess/ios/WKContentViewInteraction.mm:
(canAttemptTextRecognitionForNonImageElements):

Add a new WebKitAdditions integration point.

(-[WKContentView imageAnalysisGestureDidBegin:]):
* WebProcess/WebPage/ios/WebPageIOS.mm:
(WebKit::selectionPositionInformation):

Adjust for the changes to the built-in media controls stylesheet by adding logic to prevent the magnifier from
showing up when long pressing (or long pressing inside) video elements on iOS.

LayoutTests:

Adjust a modern media controls test, such that it no longer verifies that the `-webkit-user-select` CSS property
is none on an audio element; in lieu of this, we add a new layout test in `editing/selection` to verify that
long pressing over the timestamp of an audio element does not trigger text selection.

* editing/selection/ios/do-not-allow-text-selection-in-audio-element-expected.txt: Added.
* editing/selection/ios/do-not-allow-text-selection-in-audio-element.html: Added.
* media/modern-media-controls/audio/audio-controls-styles-expected.txt:
* media/modern-media-controls/audio/audio-controls-styles.html:</pre>

<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkLayoutTestsChangeLog">trunk/LayoutTests/ChangeLog</a></li>
<li><a href="#trunkLayoutTestsmediamodernmediacontrolsaudioaudiocontrolsstylesexpectedtxt">trunk/LayoutTests/media/modern-media-controls/audio/audio-controls-styles-expected.txt</a></li>
<li><a href="#trunkLayoutTestsmediamodernmediacontrolsaudioaudiocontrolsstyleshtml">trunk/LayoutTests/media/modern-media-controls/audio/audio-controls-styles.html</a></li>
<li><a href="#trunkSourceWebCoreChangeLog">trunk/Source/WebCore/ChangeLog</a></li>
<li><a href="#trunkSourceWebCoreModulesmediacontrolsMediaControlsHostcpp">trunk/Source/WebCore/Modules/mediacontrols/MediaControlsHost.cpp</a></li>
<li><a href="#trunkSourceWebCoreModulesmediacontrolsMediaControlsHosth">trunk/Source/WebCore/Modules/mediacontrols/MediaControlsHost.h</a></li>
<li><a href="#trunkSourceWebCoreModulesmediacontrolsMediaControlsHostidl">trunk/Source/WebCore/Modules/mediacontrols/MediaControlsHost.idl</a></li>
<li><a href="#trunkSourceWebCoreModulesmodernmediacontrolscontrolscontrolsbarcss">trunk/Source/WebCore/Modules/modern-media-controls/controls/controls-bar.css</a></li>
<li><a href="#trunkSourceWebCoreModulesmodernmediacontrolscontrolsmediacontrolscss">trunk/Source/WebCore/Modules/modern-media-controls/controls/media-controls.css</a></li>
<li><a href="#trunkSourceWebCoreModulesmodernmediacontrolsmediamediacontrollerjs">trunk/Source/WebCore/Modules/modern-media-controls/media/media-controller.js</a></li>
<li><a href="#trunkSourceWebCorehtmlHTMLElementcpp">trunk/Source/WebCore/html/HTMLElement.cpp</a></li>
<li><a href="#trunkSourceWebCorehtmlHTMLMediaElementcpp">trunk/Source/WebCore/html/HTMLMediaElement.cpp</a></li>
<li><a href="#trunkSourceWebKitChangeLog">trunk/Source/WebKit/ChangeLog</a></li>
<li><a href="#trunkSourceWebKitSharediosInteractionInformationAtPositionh">trunk/Source/WebKit/Shared/ios/InteractionInformationAtPosition.h</a></li>
<li><a href="#trunkSourceWebKitUIProcessiosWKContentViewInteractionmm">trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm</a></li>
<li><a href="#trunkSourceWebKitWebProcessWebPageiosWebPageIOSmm">trunk/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm</a></li>
</ul>

<h3>Added Paths</h3>
<ul>
<li><a href="#trunkLayoutTestseditingselectioniosdonotallowtextselectioninaudioelementexpectedtxt">trunk/LayoutTests/editing/selection/ios/do-not-allow-text-selection-in-audio-element-expected.txt</a></li>
<li><a href="#trunkLayoutTestseditingselectioniosdonotallowtextselectioninaudioelementhtml">trunk/LayoutTests/editing/selection/ios/do-not-allow-text-selection-in-audio-element.html</a></li>
</ul>

</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkLayoutTestsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/LayoutTests/ChangeLog (285608 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/ChangeLog      2021-11-10 22:49:56 UTC (rev 285608)
+++ trunk/LayoutTests/ChangeLog 2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -1,3 +1,20 @@
</span><ins>+2021-11-10  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        Refactor some image overlay logic to work with built-in media controls
+        https://bugs.webkit.org/show_bug.cgi?id=232899
+        rdar://83173597
+
+        Reviewed by Antoine Quint and Tim Horton.
+
+        Adjust a modern media controls test, such that it no longer verifies that the `-webkit-user-select` CSS property
+        is none on an audio element; in lieu of this, we add a new layout test in `editing/selection` to verify that
+        long pressing over the timestamp of an audio element does not trigger text selection.
+
+        * editing/selection/ios/do-not-allow-text-selection-in-audio-element-expected.txt: Added.
+        * editing/selection/ios/do-not-allow-text-selection-in-audio-element.html: Added.
+        * media/modern-media-controls/audio/audio-controls-styles-expected.txt:
+        * media/modern-media-controls/audio/audio-controls-styles.html:
+
</ins><span class="cx"> 2021-11-10  Antoine Quint  <graouts@webkit.org>
</span><span class="cx"> 
</span><span class="cx">         The cssText property for a computed style should return an empty string
</span></span></pre></div>
<a id="trunkLayoutTestseditingselectioniosdonotallowtextselectioninaudioelementexpectedtxt"></a>
<div class="addfile"><h4>Added: trunk/LayoutTests/editing/selection/ios/do-not-allow-text-selection-in-audio-element-expected.txt (0 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/editing/selection/ios/do-not-allow-text-selection-in-audio-element-expected.txt                                (rev 0)
+++ trunk/LayoutTests/editing/selection/ios/do-not-allow-text-selection-in-audio-element-expected.txt   2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -0,0 +1,10 @@
</span><ins>+This test verifies that long pressing the timestamp in an audio element does not start a text selection. To manually run the test, long press the audio element below and verify that no text is selected.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+PASS rects.length is 0
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
</ins></span></pre></div>
<a id="trunkLayoutTestseditingselectioniosdonotallowtextselectioninaudioelementhtml"></a>
<div class="addfile"><h4>Added: trunk/LayoutTests/editing/selection/ios/do-not-allow-text-selection-in-audio-element.html (0 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/editing/selection/ios/do-not-allow-text-selection-in-audio-element.html                                (rev 0)
+++ trunk/LayoutTests/editing/selection/ios/do-not-allow-text-selection-in-audio-element.html   2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -0,0 +1,34 @@
</span><ins>+<!DOCTYPE html> <!-- webkit-test-runner [ useFlexibleViewport=true ] -->
+<html>
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<head>
+<script src="../../../resources/js-test.js"></script>
+<script src="../../../resources/ui-helper.js"></script>
+<script>
+jsTestIsAsync = true;
+description("This test verifies that long pressing the timestamp in an audio element does not start a text selection. To manually run the test, long press the audio element below and verify that no text is selected.");
+addEventListener("load", async () => {
+    if (!window.internals)
+        return;
+
+    const audioElement = document.querySelector("audio");
+    let timeLabel;
+    for (let label of Array.from(internals.shadowRoot(audioElement).querySelectorAll("div.time-label"))) {
+        if (getComputedStyle(label).display !== "none")
+            timeLabel = label;
+    }
+
+    const targetX = (audioElement.offsetLeft + timeLabel.offsetLeft) + timeLabel.offsetWidth / 2;
+    const targetY = (audioElement.offsetTop + timeLabel.offsetTop) + timeLabel.offsetHeight / 2;
+    await UIHelper.longPressAtPoint(targetX, targetY);
+    rects = await UIHelper.getUISelectionViewRects();
+
+    shouldBe("rects.length", "0");
+    finishJSTest();
+});
+</script>
+</head>
+<body>
+<audio controls src="../../../media/content/audio-tracks.mp4"></audio>
+</body>
+</html>
</ins><span class="cx">\ No newline at end of file
</span></span></pre></div>
<a id="trunkLayoutTestsmediamodernmediacontrolsaudioaudiocontrolsstylesexpectedtxt"></a>
<div class="modfile"><h4>Modified: trunk/LayoutTests/media/modern-media-controls/audio/audio-controls-styles-expected.txt (285608 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/media/modern-media-controls/audio/audio-controls-styles-expected.txt   2021-11-10 22:49:56 UTC (rev 285608)
+++ trunk/LayoutTests/media/modern-media-controls/audio/audio-controls-styles-expected.txt      2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -3,7 +3,6 @@
</span><span class="cx"> On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
</span><span class="cx"> 
</span><span class="cx"> 
</span><del>-PASS getComputedStyle(audio).WebkitUserSelect is "none"
</del><span class="cx"> PASS getComputedStyle(audio).WebkitTouchCallout is "none"
</span><span class="cx"> 
</span><span class="cx"> PASS successfullyParsed is true
</span></span></pre></div>
<a id="trunkLayoutTestsmediamodernmediacontrolsaudioaudiocontrolsstyleshtml"></a>
<div class="modfile"><h4>Modified: trunk/LayoutTests/media/modern-media-controls/audio/audio-controls-styles.html (285608 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/media/modern-media-controls/audio/audio-controls-styles.html   2021-11-10 22:49:56 UTC (rev 285608)
+++ trunk/LayoutTests/media/modern-media-controls/audio/audio-controls-styles.html      2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -8,7 +8,6 @@
</span><span class="cx"> const audio = document.body.appendChild(document.createElement("audio"));
</span><span class="cx"> audio.controls = true;
</span><span class="cx"> 
</span><del>-shouldBeEqualToString("getComputedStyle(audio).WebkitUserSelect", "none");
</del><span class="cx"> shouldBeEqualToString("getComputedStyle(audio).WebkitTouchCallout", "none");
</span><span class="cx"> 
</span><span class="cx"> debug("");
</span></span></pre></div>
<a id="trunkSourceWebCoreChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/ChangeLog (285608 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/ChangeLog   2021-11-10 22:49:56 UTC (rev 285608)
+++ trunk/Source/WebCore/ChangeLog      2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -1,3 +1,63 @@
</span><ins>+2021-11-10  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        Refactor some image overlay logic to work with built-in media controls
+        https://bugs.webkit.org/show_bug.cgi?id=232899
+        rdar://83173597
+
+        Reviewed by Antoine Quint and Tim Horton.
+
+        Make various minor adjustments to allow built-in modern media controls to play well with image overlay content.
+        See below for more details.
+
+        * Modules/mediacontrols/MediaControlsHost.cpp:
+        (WebCore::MediaControlsHost::mediaControlsContainerClassName):
+        * Modules/mediacontrols/MediaControlsHost.h:
+
+        Add a helper function to grab the "media-controls-container" class name, which is used for the `div` element
+        containing built-in modern media controls. This is used below to identify existing media control container
+        elements when determining where to inject the image overlay root container.
+
+        * Modules/mediacontrols/MediaControlsHost.idl:
+        * Modules/modern-media-controls/controls/controls-bar.css:
+        (.controls-bar):
+
+        Z-order the media controls bar (which contains all interactible media control elements) above any image overlay
+        content that may coexist in the same shadow root.
+
+        * Modules/modern-media-controls/controls/media-controls.css:
+        (:host):
+
+        Remove `-webkit-user-select: none;` here. This was added to prevent the iOS magnifier UI from showing up when
+        long pressing inside a video element; we can achieve the same effect without applying this property over the
+        entire host element by instead changing selection logic in WebKit2.
+
+        (.media-controls):
+
+        Push the `-webkit-user-select: none;` property down into the media control children, instead of on
+        `.media-controls` itself.
+
+        (.media-controls > *):
+        * Modules/modern-media-controls/media/media-controller.js:
+        (MediaController):
+
+        Change this to ask for `mediaControlsContainerClassName` from the host, instead of hard-coding it to
+        "media-controls-container". From code inspection, there does not seem to be any codepath that passes in an
+        undefined `host`, except for the modern media controls layout tests (which still pass after this adjustment).
+
+        * html/HTMLElement.cpp:
+        (WebCore::HTMLElement::isImageOverlayText):
+        (WebCore::HTMLElement::removeImageOverlaySoonIfNeeded):
+
+        Adjust these helper methods to work in the case where the image overlay container is hosted underneath the media
+        controls container.
+
+        (WebCore::HTMLElement::updateWithTextRecognitionResult):
+        * html/HTMLMediaElement.cpp:
+        (WebCore::HTMLMediaElement::seekWithTolerance):
+        (WebCore::HTMLMediaElement::playInternal):
+
+        If needed, remove the image overlay when seeking or playing media.
+
</ins><span class="cx"> 2021-11-10  Antoine Quint  <graouts@webkit.org>
</span><span class="cx"> 
</span><span class="cx">         The cssText property for a computed style should return an empty string
</span></span></pre></div>
<a id="trunkSourceWebCoreModulesmediacontrolsMediaControlsHostcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/Modules/mediacontrols/MediaControlsHost.cpp (285608 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/Modules/mediacontrols/MediaControlsHost.cpp 2021-11-10 22:49:56 UTC (rev 285608)
+++ trunk/Source/WebCore/Modules/mediacontrols/MediaControlsHost.cpp    2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -133,6 +133,12 @@
</span><span class="cx"> #endif
</span><span class="cx"> }
</span><span class="cx"> 
</span><ins>+const AtomString& MediaControlsHost::mediaControlsContainerClassName() const
+{
+    static MainThreadNeverDestroyed<const AtomString> className("media-controls-container", AtomString::ConstructFromLiteral);
+    return className;
+}
+
</ins><span class="cx"> Vector<RefPtr<TextTrack>> MediaControlsHost::sortedTrackListForMenu(TextTrackList& trackList)
</span><span class="cx"> {
</span><span class="cx">     if (!m_mediaElement)
</span></span></pre></div>
<a id="trunkSourceWebCoreModulesmediacontrolsMediaControlsHosth"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/Modules/mediacontrols/MediaControlsHost.h (285608 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/Modules/mediacontrols/MediaControlsHost.h   2021-11-10 22:49:56 UTC (rev 285608)
+++ trunk/Source/WebCore/Modules/mediacontrols/MediaControlsHost.h      2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -55,6 +55,7 @@
</span><span class="cx">     static const AtomString& forcedOnlyKeyword();
</span><span class="cx"> 
</span><span class="cx">     String layoutTraitsClassName() const;
</span><ins>+    const AtomString& mediaControlsContainerClassName() const;
</ins><span class="cx"> 
</span><span class="cx">     double brightness() const { return 1; }
</span><span class="cx">     void setBrightness(double) { }
</span></span></pre></div>
<a id="trunkSourceWebCoreModulesmediacontrolsMediaControlsHostidl"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/Modules/mediacontrols/MediaControlsHost.idl (285608 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/Modules/mediacontrols/MediaControlsHost.idl 2021-11-10 22:49:56 UTC (rev 285608)
+++ trunk/Source/WebCore/Modules/mediacontrols/MediaControlsHost.idl    2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -35,6 +35,7 @@
</span><span class="cx">     LegacyNoInterfaceObject,
</span><span class="cx"> ] interface MediaControlsHost {
</span><span class="cx">     readonly attribute DOMString layoutTraitsClassName;
</span><ins>+    readonly attribute DOMString mediaControlsContainerClassName;
</ins><span class="cx">     attribute double brightness;
</span><span class="cx"> 
</span><span class="cx">     sequence<TextTrack> sortedTrackListForMenu(TextTrackList trackList);
</span></span></pre></div>
<a id="trunkSourceWebCoreModulesmodernmediacontrolscontrolscontrolsbarcss"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/Modules/modern-media-controls/controls/controls-bar.css (285608 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/Modules/modern-media-controls/controls/controls-bar.css     2021-11-10 22:49:56 UTC (rev 285608)
+++ trunk/Source/WebCore/Modules/modern-media-controls/controls/controls-bar.css        2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -28,4 +28,5 @@
</span><span class="cx">     cursor: default;
</span><span class="cx">     direction: ltr;
</span><span class="cx">     will-change: z-index;
</span><ins>+    z-index: 1;
</ins><span class="cx"> }
</span></span></pre></div>
<a id="trunkSourceWebCoreModulesmodernmediacontrolscontrolsmediacontrolscss"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/Modules/modern-media-controls/controls/media-controls.css (285608 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/Modules/modern-media-controls/controls/media-controls.css   2021-11-10 22:49:56 UTC (rev 285608)
+++ trunk/Source/WebCore/Modules/modern-media-controls/controls/media-controls.css      2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -24,8 +24,7 @@
</span><span class="cx">  */
</span><span class="cx"> 
</span><span class="cx"> :host {
</span><del>-    /* We always want to prevent iOS dialogs and the magnifier UI to be shown. */
-    -webkit-user-select: none !important;
</del><ins>+    /* We always want to avoid showing the context menu on iOS. */
</ins><span class="cx">     -webkit-touch-callout: none !important;
</span><span class="cx"> }
</span><span class="cx"> 
</span><span class="lines">@@ -82,7 +81,6 @@
</span><span class="cx"> .media-controls {
</span><span class="cx">     height: 100%;
</span><span class="cx">     font-family: -apple-system;
</span><del>-    -webkit-user-select: none;
</del><span class="cx">     white-space: nowrap;
</span><span class="cx">     text-align: initial;
</span><span class="cx">     transform-origin: 0 0;
</span><span class="lines">@@ -93,6 +91,8 @@
</span><span class="cx"> }
</span><span class="cx"> 
</span><span class="cx"> .media-controls > * {
</span><ins>+    /* We always want to avoid showing the text selection magnifier on iOS. */
+    -webkit-user-select: none;
</ins><span class="cx">     transition: opacity 0.1s linear;
</span><span class="cx"> }
</span><span class="cx"> 
</span><span class="lines">@@ -126,4 +126,4 @@
</span><span class="cx"> @media (inverted-colors) {
</span><span class="cx">     :host { filter: invert(100%); } /* WebKit native audio and video. */
</span><span class="cx">     picture { filter: none; } /* Don't invert the control buttons. */
</span><del>-}
</del><span class="cx">\ No newline at end of file
</span><ins>+}
</ins></span></pre></div>
<a id="trunkSourceWebCoreModulesmodernmediacontrolsmediamediacontrollerjs"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/Modules/modern-media-controls/media/media-controller.js (285608 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/Modules/modern-media-controls/media/media-controller.js     2021-11-10 22:49:56 UTC (rev 285608)
+++ trunk/Source/WebCore/Modules/modern-media-controls/media/media-controller.js        2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -37,12 +37,12 @@
</span><span class="cx">         this.hasPlayed = false;
</span><span class="cx"> 
</span><span class="cx">         this.container = shadowRoot.appendChild(document.createElement("div"));
</span><del>-        this.container.className = "media-controls-container";
</del><span class="cx"> 
</span><span class="cx">         this._updateControlsIfNeeded();
</span><span class="cx">         this._usesLTRUserInterfaceLayoutDirection = false;
</span><span class="cx"> 
</span><span class="cx">         if (host) {
</span><ins>+            this.container.className = host.mediaControlsContainerClassName;
</ins><span class="cx">             host.controlsDependOnPageScaleFactor = this.layoutTraits.controlsDependOnPageScaleFactor();
</span><span class="cx">             this.container.insertBefore(host.textTrackContainer, this.controls.element);
</span><span class="cx">             if (host.isInMediaDocument)
</span></span></pre></div>
<a id="trunkSourceWebCorehtmlHTMLElementcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/html/HTMLElement.cpp (285608 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/html/HTMLElement.cpp        2021-11-10 22:49:56 UTC (rev 285608)
+++ trunk/Source/WebCore/html/HTMLElement.cpp   2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -53,6 +53,7 @@
</span><span class="cx"> #include "HTMLFieldSetElement.h"
</span><span class="cx"> #include "HTMLFormElement.h"
</span><span class="cx"> #include "HTMLInputElement.h"
</span><ins>+#include "HTMLMediaElement.h"
</ins><span class="cx"> #include "HTMLNames.h"
</span><span class="cx"> #include "HTMLOptGroupElement.h"
</span><span class="cx"> #include "HTMLOptionElement.h"
</span><span class="lines">@@ -61,6 +62,7 @@
</span><span class="cx"> #include "HTMLStyleElement.h"
</span><span class="cx"> #include "HTMLTextAreaElement.h"
</span><span class="cx"> #include "HTMLTextFormControlElement.h"
</span><ins>+#include "MediaControlsHost.h"
</ins><span class="cx"> #include "NodeTraversal.h"
</span><span class="cx"> #include "RenderElement.h"
</span><span class="cx"> #include "RenderImage.h"
</span><span class="lines">@@ -1250,6 +1252,9 @@
</span><span class="cx">     setAttributeWithoutSynchronization(enterkeyhintAttr, value);
</span><span class="cx"> }
</span><span class="cx"> 
</span><ins>+// FIXME: We should move all of this image overlay-specific logic out into a separate helper file (possibly as standalone functions)
+// to better compartmentalize this code.
+
</ins><span class="cx"> static const AtomString& imageOverlayElementIdentifier()
</span><span class="cx"> {
</span><span class="cx">     static MainThreadNeverDestroyed<const AtomString> identifier("image-overlay", AtomString::ConstructFromLiteral);
</span><span class="lines">@@ -1323,10 +1328,8 @@
</span><span class="cx">     if (!host)
</span><span class="cx">         return false;
</span><span class="cx"> 
</span><del>-    for (auto& child : childrenOfType<HTMLDivElement>(*host->userAgentShadowRoot())) {
-        if (child.getIdAttribute() == imageOverlayElementIdentifier())
-            return node.isDescendantOf(child);
-    }
</del><ins>+    if (RefPtr overlay = static_cast<TreeScope&>(*host->userAgentShadowRoot()).getElementById(imageOverlayElementIdentifier()))
+        return node.isDescendantOf(*overlay);
</ins><span class="cx"> 
</span><span class="cx">     return false;
</span><span class="cx"> }
</span><span class="lines">@@ -1345,17 +1348,9 @@
</span><span class="cx">         if (!shadowRoot)
</span><span class="cx">             return;
</span><span class="cx"> 
</span><del>-        RefPtr<HTMLDivElement> containerToRemove;
-        for (auto& child : childrenOfType<HTMLDivElement>(*shadowRoot)) {
-            if (child.getIdAttribute() == imageOverlayElementIdentifier()) {
-                containerToRemove = &child;
-                break;
-            }
-        }
</del><ins>+        if (RefPtr overlay = static_cast<TreeScope&>(*shadowRoot).getElementById(imageOverlayElementIdentifier()))
+            overlay->remove();
</ins><span class="cx"> 
</span><del>-        if (containerToRemove)
-            containerToRemove->remove();
-
</del><span class="cx"> #if ENABLE(IMAGE_ANALYSIS)
</span><span class="cx">         if (auto page = protectedThis->document().page())
</span><span class="cx">             page->resetTextRecognitionResult(*protectedThis);
</span><span class="lines">@@ -1395,15 +1390,35 @@
</span><span class="cx"> 
</span><span class="cx">     bool hadExistingTextRecognitionElements = false;
</span><span class="cx">     TextRecognitionElements textRecognitionElements;
</span><del>-
-    if (hasImageOverlay()) {
-        for (auto& child : childrenOfType<HTMLDivElement>(*userAgentShadowRoot())) {
-            if (child.getIdAttribute() == imageOverlayElementIdentifier()) {
-                textRecognitionElements.root = &child;
-                hadExistingTextRecognitionElements = true;
-                break;
</del><ins>+    RefPtr<HTMLElement> mediaControlsContainer;
+    if (RefPtr shadowRoot = this->shadowRoot()) {
+#if ENABLE(MODERN_MEDIA_CONTROLS)
+        if (is<HTMLMediaElement>(*this)) {
+            if (RefPtr controlsHost = downcast<HTMLMediaElement>(*this).mediaControlsHost()) {
+                auto& containerClass = controlsHost->mediaControlsContainerClassName();
+                for (auto& child : childrenOfType<HTMLDivElement>(*shadowRoot)) {
+                    if (child.hasClass() && child.classNames().contains(containerClass)) {
+                        mediaControlsContainer = &child;
+                        break;
+                    }
+                }
</ins><span class="cx">             }
</span><span class="cx">         }
</span><ins>+#endif
+        if (hasImageOverlay()) {
+            RefPtr<ContainerNode> containerForImageOverlay;
+            if (mediaControlsContainer)
+                containerForImageOverlay = mediaControlsContainer;
+            else
+                containerForImageOverlay = shadowRoot;
+            for (auto& child : childrenOfType<HTMLDivElement>(*containerForImageOverlay)) {
+                if (child.getIdAttribute() == imageOverlayElementIdentifier()) {
+                    textRecognitionElements.root = &child;
+                    hadExistingTextRecognitionElements = true;
+                    continue;
+                }
+            }
+        }
</ins><span class="cx">     }
</span><span class="cx"> 
</span><span class="cx">     if (textRecognitionElements.root) {
</span><span class="lines">@@ -1457,7 +1472,11 @@
</span><span class="cx">         rootContainer->setIdAttribute(imageOverlayElementIdentifier());
</span><span class="cx">         if (document().isImageDocument())
</span><span class="cx">             rootContainer->setInlineStyleProperty(CSSPropertyWebkitUserSelect, CSSValueText);
</span><del>-        shadowRoot->appendChild(rootContainer);
</del><ins>+
+        if (mediaControlsContainer)
+            mediaControlsContainer->appendChild(rootContainer);
+        else
+            shadowRoot->appendChild(rootContainer);
</ins><span class="cx">         textRecognitionElements.root = rootContainer.copyRef();
</span><span class="cx">         textRecognitionElements.lines.reserveInitialCapacity(result.lines.size());
</span><span class="cx">         for (auto& line : result.lines) {
</span></span></pre></div>
<a id="trunkSourceWebCorehtmlHTMLMediaElementcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/html/HTMLMediaElement.cpp (285608 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/html/HTMLMediaElement.cpp   2021-11-10 22:49:56 UTC (rev 285608)
+++ trunk/Source/WebCore/html/HTMLMediaElement.cpp      2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -3139,6 +3139,8 @@
</span><span class="cx"> 
</span><span class="cx">     if (processingUserGestureForMedia())
</span><span class="cx">         mediaSession().removeBehaviorRestriction(MediaElementSession::RequireUserGestureToControlControlsManager);
</span><ins>+
+    removeImageOverlaySoonIfNeeded();
</ins><span class="cx"> }
</span><span class="cx"> 
</span><span class="cx"> void HTMLMediaElement::seekTask()
</span><span class="lines">@@ -3731,6 +3733,8 @@
</span><span class="cx"> 
</span><span class="cx">     m_autoplaying = false;
</span><span class="cx">     updatePlayState();
</span><ins>+
+    removeImageOverlaySoonIfNeeded();
</ins><span class="cx"> }
</span><span class="cx"> 
</span><span class="cx"> void HTMLMediaElement::pause()
</span></span></pre></div>
<a id="trunkSourceWebKitChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit/ChangeLog (285608 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit/ChangeLog    2021-11-10 22:49:56 UTC (rev 285608)
+++ trunk/Source/WebKit/ChangeLog       2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -1,3 +1,26 @@
</span><ins>+2021-11-10  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        Refactor some image overlay logic to work with built-in media controls
+        https://bugs.webkit.org/show_bug.cgi?id=232899
+        rdar://83173597
+
+        Reviewed by Antoine Quint and Tim Horton.
+
+        See WebCore/ChangeLog for more details.
+
+        * Shared/ios/InteractionInformationAtPosition.h:
+        * UIProcess/ios/WKContentViewInteraction.mm:
+        (canAttemptTextRecognitionForNonImageElements):
+
+        Add a new WebKitAdditions integration point.
+
+        (-[WKContentView imageAnalysisGestureDidBegin:]):
+        * WebProcess/WebPage/ios/WebPageIOS.mm:
+        (WebKit::selectionPositionInformation):
+
+        Adjust for the changes to the built-in media controls stylesheet by adding logic to prevent the magnifier from
+        showing up when long pressing (or long pressing inside) video elements on iOS.
+
</ins><span class="cx"> 2021-11-10  Per Arne Vollan <pvollan@apple.com>
</span><span class="cx"> 
</span><span class="cx">         [macOS][GPUP] Remove access to sysctl properties
</span></span></pre></div>
<a id="trunkSourceWebKitSharediosInteractionInformationAtPositionh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit/Shared/ios/InteractionInformationAtPosition.h (285608 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit/Shared/ios/InteractionInformationAtPosition.h        2021-11-10 22:49:56 UTC (rev 285608)
+++ trunk/Source/WebKit/Shared/ios/InteractionInformationAtPosition.h   2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -59,6 +59,7 @@
</span><span class="cx">         UnselectableDueToFocusableElement,
</span><span class="cx">         UnselectableDueToLargeElementBounds,
</span><span class="cx">         UnselectableDueToUserSelectNone,
</span><ins>+        UnselectableDueToMediaControls,
</ins><span class="cx">     };
</span><span class="cx">     Selectability selectability { Selectability::Selectable };
</span><span class="cx"> 
</span><span class="lines">@@ -132,7 +133,8 @@
</span><span class="cx">         WebKit::InteractionInformationAtPosition::Selectability::Selectable,
</span><span class="cx">         WebKit::InteractionInformationAtPosition::Selectability::UnselectableDueToFocusableElement,
</span><span class="cx">         WebKit::InteractionInformationAtPosition::Selectability::UnselectableDueToLargeElementBounds,
</span><del>-        WebKit::InteractionInformationAtPosition::Selectability::UnselectableDueToUserSelectNone
</del><ins>+        WebKit::InteractionInformationAtPosition::Selectability::UnselectableDueToUserSelectNone,
+        WebKit::InteractionInformationAtPosition::Selectability::UnselectableDueToMediaControls
</ins><span class="cx">     >;
</span><span class="cx"> };
</span><span class="cx"> 
</span></span></pre></div>
<a id="trunkSourceWebKitUIProcessiosWKContentViewInteractionmm"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm (285608 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm    2021-11-10 22:49:56 UTC (rev 285608)
+++ trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm       2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -226,6 +226,19 @@
</span><span class="cx"> 
</span><span class="cx"> #endif // ENABLE(IMAGE_ANALYSIS)
</span><span class="cx"> 
</span><ins>+#if USE(APPLE_INTERNAL_SDK)
+#import <WebKitAdditions/WKContentViewInteractionAdditions.mm>
+#else
+
+#if ENABLE(IMAGE_ANALYSIS)
+static bool canAttemptTextRecognitionForNonImageElements(const WebKit::InteractionInformationAtPosition& information, const WebKit::WebPreferences&)
+{
+    return false;
+}
+#endif // ENABLE(IMAGE_ANALYSIS)
+
+#endif
+
</ins><span class="cx"> namespace WebKit {
</span><span class="cx"> using namespace WebCore;
</span><span class="cx"> using namespace WebKit;
</span><span class="lines">@@ -10187,7 +10200,7 @@
</span><span class="cx">             return;
</span><span class="cx"> 
</span><span class="cx">         bool shouldAnalyzeImageAtLocation = ([&] {
</span><del>-            if (!information.isImage)
</del><ins>+            if (!information.isImage && !canAttemptTextRecognitionForNonImageElements(information, strongSelf->_page->preferences()))
</ins><span class="cx">                 return false;
</span><span class="cx"> 
</span><span class="cx">             if (!information.image)
</span></span></pre></div>
<a id="trunkSourceWebKitWebProcessWebPageiosWebPageIOSmm"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm (285608 => 285609)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm 2021-11-10 22:49:56 UTC (rev 285608)
+++ trunk/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm    2021-11-11 00:03:06 UTC (rev 285609)
</span><span class="lines">@@ -2940,6 +2940,9 @@
</span><span class="cx">                 // FIXME: Is this heuristic still needed, now that block selection has been removed?
</span><span class="cx">                 return InteractionInformationAtPosition::Selectability::UnselectableDueToLargeElementBounds;
</span><span class="cx">             }
</span><ins>+
+            if (hostVideoElementIgnoringImageOverlay(*hitNode))
+                return InteractionInformationAtPosition::Selectability::UnselectableDueToMediaControls;
</ins><span class="cx">         }
</span><span class="cx"> 
</span><span class="cx">         return InteractionInformationAtPosition::Selectability::Selectable;
</span></span></pre>
</div>
</div>

</body>
</html>