<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[277726] trunk</title>
</head>
<body>

<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt;  }
#msg dl a { font-weight: bold}
#msg dl a:link    { color:#fc3; }
#msg dl a:active  { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff  {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/277726">277726</a></dd>
<dt>Author</dt> <dd>jer.noble@apple.com</dd>
<dt>Date</dt> <dd>2021-05-19 09:43:28 -0700 (Wed, 19 May 2021)</dd>
</dl>

<h3>Log Message</h3>
<pre>[Mac] MSE-based video pauses when put in background; PiP context menu disabled
https://bugs.webkit.org/show_bug.cgi?id=225958
<rdar://78130303>

Reviewed by Eric Carlson.

Source/WebCore:

Test: media/media-source/media-source-has-audio-video.html

In <a href="http://trac.webkit.org/projects/webkit/changeset/277116">r277116</a>, in order to reduce the CPU usage incurred when pushing state changes from
the GPU process to the WebContent process, state changes to `hasAudio` and `hasVideo`
were now only updated when the MediaPlayer indicated that its `characteristicsChanged()`.
However, in SourceBufferPrivateAVFObjC, this call was performed too early, before the
SourceBuffer in the WebContent process had informed SourceBufferPrivate of newly added
audio and video tracks. Thus the cached values of `hasAudio` and `hasVideo` inside the
WebContent process were never updated.

Call `characteristicsChanged()` from the callback handler of `didReceiveInitializationSegment()`,
which will be called only after the WebContent process completes its handling of the
new initialization segment, which includes telling the SourceBufferPrivate in the GPU
process about newly added audio and video tracks.

* platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
(WebCore::SourceBufferPrivateAVFObjC::didParseInitializationData):

LayoutTests:

* media/media-source/media-source-has-audio-video-expected.txt: Added.
* media/media-source/media-source-has-audio-video.html: Added.</pre>

<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkLayoutTestsChangeLog">trunk/LayoutTests/ChangeLog</a></li>
<li><a href="#trunkSourceWebCoreChangeLog">trunk/Source/WebCore/ChangeLog</a></li>
<li><a href="#trunkSourceWebCoreplatformgraphicsavfoundationobjcSourceBufferPrivateAVFObjCmm">trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm</a></li>
</ul>

<h3>Added Paths</h3>
<ul>
<li><a href="#trunkLayoutTestsmediamediasourcemediasourcehasaudiovideoexpectedtxt">trunk/LayoutTests/media/media-source/media-source-has-audio-video-expected.txt</a></li>
<li><a href="#trunkLayoutTestsmediamediasourcemediasourcehasaudiovideohtml">trunk/LayoutTests/media/media-source/media-source-has-audio-video.html</a></li>
</ul>

</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkLayoutTestsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/LayoutTests/ChangeLog (277725 => 277726)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/ChangeLog      2021-05-19 16:34:52 UTC (rev 277725)
+++ trunk/LayoutTests/ChangeLog 2021-05-19 16:43:28 UTC (rev 277726)
</span><span class="lines">@@ -1,3 +1,14 @@
</span><ins>+2021-05-19  Jer Noble  <jer.noble@apple.com>
+
+        [Mac] MSE-based video pauses when put in background; PiP context menu disabled
+        https://bugs.webkit.org/show_bug.cgi?id=225958
+        <rdar://78130303>
+
+        Reviewed by Eric Carlson.
+
+        * media/media-source/media-source-has-audio-video-expected.txt: Added.
+        * media/media-source/media-source-has-audio-video.html: Added.
+
</ins><span class="cx"> 2021-05-18  Cameron McCormack  <heycam@apple.com>
</span><span class="cx"> 
</span><span class="cx">         Record gradient and pattern filled canvas text in the correct coordinate system.
</span></span></pre></div>
<a id="trunkLayoutTestsmediamediasourcemediasourcehasaudiovideoexpectedtxt"></a>
<div class="addfile"><h4>Added: trunk/LayoutTests/media/media-source/media-source-has-audio-video-expected.txt (0 => 277726)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/media/media-source/media-source-has-audio-video-expected.txt                           (rev 0)
+++ trunk/LayoutTests/media/media-source/media-source-has-audio-video-expected.txt      2021-05-19 16:43:28 UTC (rev 277726)
</span><span class="lines">@@ -0,0 +1,11 @@
</span><ins>+
+RUN(video.src = URL.createObjectURL(mediaSource))
+EVENT(sourceopen)
+RUN(mediaSource.duration = loader.duration())
+RUN(sourceBuffer = mediaSource.addSourceBuffer(loader.type()))
+RUN(sourceBuffer.appendBuffer(loader.initSegment()))
+EVENT(update)
+EXPECTED (internals.mediaUsageState(video).hasVideo == 'true') OK
+EXPECTED (internals.mediaUsageState(video).hasAudio == 'true') OK
+END OF TEST
+
</ins></span></pre></div>
<a id="trunkLayoutTestsmediamediasourcemediasourcehasaudiovideohtml"></a>
<div class="addfile"><h4>Added: trunk/LayoutTests/media/media-source/media-source-has-audio-video.html (0 => 277726)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/media/media-source/media-source-has-audio-video.html                           (rev 0)
+++ trunk/LayoutTests/media/media-source/media-source-has-audio-video.html      2021-05-19 16:43:28 UTC (rev 277726)
</span><span class="lines">@@ -0,0 +1,43 @@
</span><ins>+<!DOCTYPE html>
+<html>
+<head>
+    <title>media-source-has-audio-video</title>
+    <script src="media-source-loader.js"></script>
+    <script src="../video-test.js"></script>
+    <script>
+    window.addEventListener('load', event => {
+        findMediaElement();
+
+        window.sourceBuffer = new MediaSource();
+
+        loader = new MediaSourceLoader('content/test-fragmented-manifest.json');
+        loader.onload = mediaDataLoaded;
+        loader.onerror = mediaDataLoadingFailed;
+    });
+
+    function mediaDataLoadingFailed() {
+        failTest('Media data loading failed');
+    }
+
+    async function mediaDataLoaded() {
+        window.mediaSource = new MediaSource();
+        run('video.src = URL.createObjectURL(mediaSource)');
+        waitFor(video, 'error').then(failTest);
+        await waitFor(mediaSource, 'sourceopen');
+
+        run('mediaSource.duration = loader.duration()');
+        run('sourceBuffer = mediaSource.addSourceBuffer(loader.type())');
+        run('sourceBuffer.appendBuffer(loader.initSegment())');
+        await waitFor(sourceBuffer, 'update');
+
+        await testExpectedEventually('internals.mediaUsageState(video).hasVideo', true);
+        await testExpectedEventually('internals.mediaUsageState(video).hasAudio', true);
+        endTest();
+    }
+
+    </script>
+</head>
+<body>
+    <video></video>
+</body>
+</html>
</ins></span></pre></div>
<a id="trunkSourceWebCoreChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/ChangeLog (277725 => 277726)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/ChangeLog   2021-05-19 16:34:52 UTC (rev 277725)
+++ trunk/Source/WebCore/ChangeLog      2021-05-19 16:43:28 UTC (rev 277726)
</span><span class="lines">@@ -1,3 +1,29 @@
</span><ins>+2021-05-19  Jer Noble  <jer.noble@apple.com>
+
+        [Mac] MSE-based video pauses when put in background; PiP context menu disabled
+        https://bugs.webkit.org/show_bug.cgi?id=225958
+        <rdar://78130303>
+
+        Reviewed by Eric Carlson.
+
+        Test: media/media-source/media-source-has-audio-video.html
+
+        In r277116, in order to reduce the CPU usage incurred when pushing state changes from
+        the GPU process to the WebContent process, state changes to `hasAudio` and `hasVideo`
+        were now only updated when the MediaPlayer indicated that its `characteristicsChanged()`.
+        However, in SourceBufferPrivateAVFObjC, this call was performed too early, before the
+        SourceBuffer in the WebContent process had informed SourceBufferPrivate of newly added
+        audio and video tracks. Thus the cached values of `hasAudio` and `hasVideo` inside the
+        WebContent process were never updated.
+
+        Call `characteristicsChanged()` from the callback handler of `didReceiveInitializationSegment()`,
+        which will be called only after the WebContent process completes its handling of the
+        new initialization segment, which includes telling the SourceBufferPrivate in the GPU
+        process about newly added audio and video tracks.
+
+        * platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
+        (WebCore::SourceBufferPrivateAVFObjC::didParseInitializationData):
+
</ins><span class="cx"> 2021-05-19  Antti Koivisto  <antti@apple.com>
</span><span class="cx"> 
</span><span class="cx">         Factor pseudo class invalidation code in Document::updateHoverActiveState into a lambda
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformgraphicsavfoundationobjcSourceBufferPrivateAVFObjCmm"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm (277725 => 277726)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm   2021-05-19 16:34:52 UTC (rev 277725)
+++ trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm      2021-05-19 16:43:28 UTC (rev 277726)
</span><span class="lines">@@ -409,6 +409,9 @@
</span><span class="cx">         if (!weakThis)
</span><span class="cx">             return;
</span><span class="cx"> 
</span><ins>+        if (auto player = this->player())
+            player->characteristicsChanged();
+
</ins><span class="cx">         if  (m_mediaSamples.isEmpty()) {
</span><span class="cx">             m_initializationSegmentIsHandled = true;
</span><span class="cx">             ALWAYS_LOG(LOGIDENTIFIER, "initialization segment is handled");
</span></span></pre>
</div>
</div>

</body>
</html>