<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[278004] trunk</title>
</head>
<body>

<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt;  }
#msg dl a { font-weight: bold}
#msg dl a:link    { color:#fc3; }
#msg dl a:active  { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff  {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/278004">278004</a></dd>
<dt>Author</dt> <dd>eocanha@igalia.com</dd>
<dt>Date</dt> <dd>2021-05-25 05:52:03 -0700 (Tue, 25 May 2021)</dd>
</dl>

<h3>Log Message</h3>
<pre>[GTK] Layout test media/track/track-cue-missing.html is failing
https://bugs.webkit.org/show_bug.cgi?id=191004

Reviewed by Alicia Boya Garcia.

Source/WebCore:

Implemented the missing performTaskAtMediaTime() method, needed by HTMLMediaElement::updateActiveTextTrackCues()
to trigger custom updates at times that don't fit in the regular updates statically scheduled every 250 ms.
The implementation uses triggerRepaint() for synchronization instead of timers, computing the current stream time
of the last video frame to be painted as an heuristic, instead of asking the position to the pipeline (more
expensive and risky from a non-main thread). If the stream time is beyond the task target time, the task is run
on the main thread.

Still, that is not enough. The task scheduled by HTMLMediaElement requires currentTime() to return an accurate
value, not one cached every 200ms. m_cachedTime needs to be updated more frequently (at least once every main
loop iteration), but still remain consistent along the whole iteration. This is achieved by enqueing an
invalidation task (to be run at the end of the current iteration) each time the cached value is set with a value
coming from a position query.

The changes in the m_cachedTime management add an overhead between 173% (track-cue-missing test) and 426%
(regular video playback without subtitles) in the run time spent querying the position to the pipeline. As an
optimization, the position is now directly queried to the audio and video sink and the highest value (according
to the current playback direction) is returned. This reduces the overhead to just 63% in the regular playback
without subtitles case.

The inner TaskAtMediaTimeScheduler class is used for task target time evaluation. As it can be used from the
main thread and from a secondary one in triggerRepaint(), it's wrapped in a DataMutex to manage concurrent
access. The original task is returned to the caller of checkTaskForScheduling() when the currentTime is beyond
the target time. The caller is then responsible of running the task externally in the main thread, out of the
influence of the DataMutex lock (the task can be expensive and the lock is only needed to evaluate the triggering
conditions).

Covered by existing tests.

* platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:
(WebCore::MediaPlayerPrivateGStreamer::TaskAtMediaTimeScheduler::setTask): Configure the object with the right task, targetTime, and current playback direction.
(WebCore::MediaPlayerPrivateGStreamer::TaskAtMediaTimeScheduler::checkTaskForScheduling): Check if the task must run. In that case, return the task as result so the caller can run it externally on the main thread after the DataMutex lock has been released.
(WebCore::MediaPlayerPrivateGStreamer::invalidateCachedPosition): Implementation moved to the cpp file.

* platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
(WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Don't initialize m_cachedPosition, as it's now an Optional that will be initialized as no-value.
(WebCore::MediaPlayerPrivateGStreamer::playbackPosition const): Change the time based caching mechanism to a mainloop iteration based one. Directly query the sinks instead of the pipeline for position.
(WebCore::MediaPlayerPrivateGStreamer::asyncStateChangeDone): Invalidate the Optional, instead of setting an invalid MediaTime.
(WebCore::MediaPlayerPrivateGStreamer::updateStates): Ditto.
(WebCore::MediaPlayerPrivateGStreamer::didEnd): Ditto.
(WebCore::MediaPlayerPrivateGStreamer::performTaskAtMediaTime): Set the task on the TaskAtMediaTimeScheduler when the media is currently playing. That task will run at the target time. The HTMLMediaElement periodic update will already take care if the media isn't currently playing.
(WebCore::MediaPlayerPrivateGStreamer::invalidateCachedPosition const): Reset Optional<MediaTime> m_cachedTime.
(WebCore::MediaPlayerPrivateGStreamer::invalidateCachedPositionOnNextIteration const): Enqueue an invalidation on the next mainloop iteration.
(WebCore::MediaPlayerPrivateGStreamer::triggerRepaint): Check if the PendingTask needs to be run, using the streaming time of the last rendered sample as an approximate way to get the position, and run it if that's the case.

LayoutTests:

* platform/glib/TestExpectations: Unskipped test.</pre>

<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkLayoutTestsChangeLog">trunk/LayoutTests/ChangeLog</a></li>
<li><a href="#trunkLayoutTestsplatformglibTestExpectations">trunk/LayoutTests/platform/glib/TestExpectations</a></li>
<li><a href="#trunkSourceWebCoreChangeLog">trunk/Source/WebCore/ChangeLog</a></li>
<li><a href="#trunkSourceWebCoreplatformgraphicsgstreamerMediaPlayerPrivateGStreamercpp">trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp</a></li>
<li><a href="#trunkSourceWebCoreplatformgraphicsgstreamerMediaPlayerPrivateGStreamerh">trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h</a></li>
</ul>

</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkLayoutTestsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/LayoutTests/ChangeLog (278003 => 278004)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/ChangeLog      2021-05-25 08:33:30 UTC (rev 278003)
+++ trunk/LayoutTests/ChangeLog 2021-05-25 12:52:03 UTC (rev 278004)
</span><span class="lines">@@ -1,3 +1,12 @@
</span><ins>+2021-05-25  Enrique Ocaña González  <eocanha@igalia.com>
+
+        [GTK] Layout test media/track/track-cue-missing.html is failing
+        https://bugs.webkit.org/show_bug.cgi?id=191004
+
+        Reviewed by Alicia Boya Garcia.
+
+        * platform/glib/TestExpectations: Unskipped test.
+
</ins><span class="cx"> 2021-05-25  Tim Nguyen  <ntim@apple.com>
</span><span class="cx"> 
</span><span class="cx">         Add basic <dialog> element UA styles
</span></span></pre></div>
<a id="trunkLayoutTestsplatformglibTestExpectations"></a>
<div class="modfile"><h4>Modified: trunk/LayoutTests/platform/glib/TestExpectations (278003 => 278004)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/LayoutTests/platform/glib/TestExpectations 2021-05-25 08:33:30 UTC (rev 278003)
+++ trunk/LayoutTests/platform/glib/TestExpectations    2021-05-25 12:52:03 UTC (rev 278004)
</span><span class="lines">@@ -2287,8 +2287,6 @@
</span><span class="cx"> webkit.org/b/186664 media/video-playsinline.html [ Failure ]
</span><span class="cx"> webkit.org/b/186679 media/video-currentTime-delay.html [ Crash Pass Timeout ]
</span><span class="cx"> 
</span><del>-webkit.org/b/191004 media/track/track-cue-missing.html [ Failure ]
-
</del><span class="cx"> webkit.org/b/198827 media/media-fullscreen-return-to-inline.html [ Timeout Pass Crash ]
</span><span class="cx"> webkit.org/b/198827 media/video-ended-event-slow-motion-playback.html [ Timeout Pass ]
</span><span class="cx"> 
</span></span></pre></div>
<a id="trunkSourceWebCoreChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/ChangeLog (278003 => 278004)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/ChangeLog   2021-05-25 08:33:30 UTC (rev 278003)
+++ trunk/Source/WebCore/ChangeLog      2021-05-25 12:52:03 UTC (rev 278004)
</span><span class="lines">@@ -1,3 +1,54 @@
</span><ins>+2021-05-25  Enrique Ocaña González  <eocanha@igalia.com>
+
+        [GTK] Layout test media/track/track-cue-missing.html is failing
+        https://bugs.webkit.org/show_bug.cgi?id=191004
+
+        Reviewed by Alicia Boya Garcia.
+
+        Implemented the missing performTaskAtMediaTime() method, needed by HTMLMediaElement::updateActiveTextTrackCues()
+        to trigger custom updates at times that don't fit in the regular updates statically scheduled every 250 ms.
+        The implementation uses triggerRepaint() for synchronization instead of timers, computing the current stream time
+        of the last video frame to be painted as an heuristic, instead of asking the position to the pipeline (more
+        expensive and risky from a non-main thread). If the stream time is beyond the task target time, the task is run
+        on the main thread.
+
+        Still, that is not enough. The task scheduled by HTMLMediaElement requires currentTime() to return an accurate
+        value, not one cached every 200ms. m_cachedTime needs to be updated more frequently (at least once every main
+        loop iteration), but still remain consistent along the whole iteration. This is achieved by enqueing an
+        invalidation task (to be run at the end of the current iteration) each time the cached value is set with a value
+        coming from a position query.
+
+        The changes in the m_cachedTime management add an overhead between 173% (track-cue-missing test) and 426%
+        (regular video playback without subtitles) in the run time spent querying the position to the pipeline. As an
+        optimization, the position is now directly queried to the audio and video sink and the highest value (according
+        to the current playback direction) is returned. This reduces the overhead to just 63% in the regular playback
+        without subtitles case.
+
+        The inner TaskAtMediaTimeScheduler class is used for task target time evaluation. As it can be used from the
+        main thread and from a secondary one in triggerRepaint(), it's wrapped in a DataMutex to manage concurrent
+        access. The original task is returned to the caller of checkTaskForScheduling() when the currentTime is beyond
+        the target time. The caller is then responsible of running the task externally in the main thread, out of the
+        influence of the DataMutex lock (the task can be expensive and the lock is only needed to evaluate the triggering
+        conditions).
+
+        Covered by existing tests.
+
+        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:
+        (WebCore::MediaPlayerPrivateGStreamer::TaskAtMediaTimeScheduler::setTask): Configure the object with the right task, targetTime, and current playback direction.
+        (WebCore::MediaPlayerPrivateGStreamer::TaskAtMediaTimeScheduler::checkTaskForScheduling): Check if the task must run. In that case, return the task as result so the caller can run it externally on the main thread after the DataMutex lock has been released.
+        (WebCore::MediaPlayerPrivateGStreamer::invalidateCachedPosition): Implementation moved to the cpp file.
+
+        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
+        (WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer): Don't initialize m_cachedPosition, as it's now an Optional that will be initialized as no-value.
+        (WebCore::MediaPlayerPrivateGStreamer::playbackPosition const): Change the time based caching mechanism to a mainloop iteration based one. Directly query the sinks instead of the pipeline for position.
+        (WebCore::MediaPlayerPrivateGStreamer::asyncStateChangeDone): Invalidate the Optional, instead of setting an invalid MediaTime.
+        (WebCore::MediaPlayerPrivateGStreamer::updateStates): Ditto.
+        (WebCore::MediaPlayerPrivateGStreamer::didEnd): Ditto.
+        (WebCore::MediaPlayerPrivateGStreamer::performTaskAtMediaTime): Set the task on the TaskAtMediaTimeScheduler when the media is currently playing. That task will run at the target time. The HTMLMediaElement periodic update will already take care if the media isn't currently playing.
+        (WebCore::MediaPlayerPrivateGStreamer::invalidateCachedPosition const): Reset Optional<MediaTime> m_cachedTime.
+        (WebCore::MediaPlayerPrivateGStreamer::invalidateCachedPositionOnNextIteration const): Enqueue an invalidation on the next mainloop iteration.
+        (WebCore::MediaPlayerPrivateGStreamer::triggerRepaint): Check if the PendingTask needs to be run, using the streaming time of the last rendered sample as an approximate way to get the position, and run it if that's the case.
+
</ins><span class="cx"> 2021-05-25  Tim Nguyen  <ntim@apple.com>
</span><span class="cx"> 
</span><span class="cx">         Add basic <dialog> element UA styles
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformgraphicsgstreamerMediaPlayerPrivateGStreamercpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp (278003 => 278004)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp 2021-05-25 08:33:30 UTC (rev 278003)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp    2021-05-25 12:52:03 UTC (rev 278004)
</span><span class="lines">@@ -146,7 +146,6 @@
</span><span class="cx">     : m_notifier(MainThreadNotifier<MainThreadNotification>::create())
</span><span class="cx">     , m_player(player)
</span><span class="cx">     , m_referrer(player->referrer())
</span><del>-    , m_cachedPosition(MediaTime::invalidTime())
</del><span class="cx">     , m_cachedDuration(MediaTime::invalidTime())
</span><span class="cx">     , m_seekTime(MediaTime::invalidTime())
</span><span class="cx">     , m_timeOfOverlappingSeek(MediaTime::invalidTime())
</span><span class="lines">@@ -1344,23 +1343,32 @@
</span><span class="cx">     if (m_isEndReached)
</span><span class="cx">         return m_playbackRate > 0 ? durationMediaTime() : MediaTime::zeroTime();
</span><span class="cx"> 
</span><del>-    // This constant should remain lower than HTMLMediaElement's maxTimeupdateEventFrequency.
-    static const Seconds positionCacheThreshold = 200_ms;
-    Seconds now = WTF::WallTime::now().secondsSinceEpoch();
-    if (m_lastQueryTime && (now - m_lastQueryTime.value()) < positionCacheThreshold && m_cachedPosition.isValid()) {
-        GST_TRACE_OBJECT(pipeline(), "Returning cached position: %s", m_cachedPosition.toString().utf8().data());
-        return m_cachedPosition;
</del><ins>+    if (m_cachedPosition.hasValue()) {
+        GST_TRACE_OBJECT(pipeline(), "Returning cached position: %s", m_cachedPosition.value().toString().utf8().data());
+        return m_cachedPosition.value();
</ins><span class="cx">     }
</span><span class="cx"> 
</span><del>-    m_lastQueryTime = now;
-
</del><span class="cx">     // Position is only available if no async state change is going on and the state is either paused or playing.
</span><span class="cx">     gint64 position = GST_CLOCK_TIME_NONE;
</span><del>-    GstQuery* query = gst_query_new_position(GST_FORMAT_TIME);
-    if (gst_element_query(m_pipeline.get(), query))
-        gst_query_parse_position(query, 0, &position);
-    gst_query_unref(query);
</del><span class="cx"> 
</span><ins>+    // Asking directly to the sinks and choosing the highest value is faster than asking to the pipeline.
+    GRefPtr<GstQuery> query = adoptGRef(gst_query_new_position(GST_FORMAT_TIME));
+    if (m_audioSink && gst_element_query(m_audioSink.get(), query.get())) {
+        gint64 audioPosition = GST_CLOCK_TIME_NONE;
+        gst_query_parse_position(query.get(), 0, &audioPosition);
+        if (GST_CLOCK_TIME_IS_VALID(audioPosition))
+            position = audioPosition;
+        query = adoptGRef(gst_query_new_position(GST_FORMAT_TIME));
+    }
+    if (m_videoSink && gst_element_query(m_videoSink.get(), query.get())) {
+        gint64 videoPosition = GST_CLOCK_TIME_NONE;
+        gst_query_parse_position(query.get(), 0, &videoPosition);
+        if (GST_CLOCK_TIME_IS_VALID(videoPosition) && (!GST_CLOCK_TIME_IS_VALID(position)
+            || (m_playbackRate >= 0 && videoPosition > position)
+            || (m_playbackRate < 0 && videoPosition < position)))
+            position = videoPosition;
+    }
+
</ins><span class="cx">     GstClockTime gstreamerPosition = static_cast<GstClockTime>(position);
</span><span class="cx">     GST_TRACE_OBJECT(pipeline(), "Position %" GST_TIME_FORMAT ", canFallBackToLastFinishedSeekPosition: %s", GST_TIME_ARGS(gstreamerPosition), boolForPrinting(m_canFallBackToLastFinishedSeekPosition));
</span><span class="cx"> 
</span><span class="lines">@@ -1372,6 +1380,7 @@
</span><span class="cx">         playbackPosition = m_seekTime;
</span><span class="cx"> 
</span><span class="cx">     m_cachedPosition = playbackPosition;
</span><ins>+    invalidateCachedPositionOnNextIteration();
</ins><span class="cx">     return playbackPosition;
</span><span class="cx"> }
</span><span class="cx"> 
</span><span class="lines">@@ -2235,7 +2244,7 @@
</span><span class="cx">         else {
</span><span class="cx">             GST_DEBUG_OBJECT(pipeline(), "[Seek] seeked to %s", toString(m_seekTime).utf8().data());
</span><span class="cx">             m_isSeeking = false;
</span><del>-            m_cachedPosition = MediaTime::invalidTime();
</del><ins>+            invalidateCachedPosition();
</ins><span class="cx">             if (m_timeOfOverlappingSeek != m_seekTime && m_timeOfOverlappingSeek.isValid()) {
</span><span class="cx">                 seek(m_timeOfOverlappingSeek);
</span><span class="cx">                 m_timeOfOverlappingSeek = MediaTime::invalidTime();
</span><span class="lines">@@ -2421,7 +2430,7 @@
</span><span class="cx">             m_isSeekPending = false;
</span><span class="cx">             m_isSeeking = doSeek(m_seekTime, m_player->rate(), static_cast<GstSeekFlags>(GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_ACCURATE));
</span><span class="cx">             if (!m_isSeeking) {
</span><del>-                m_cachedPosition = MediaTime::invalidTime();
</del><ins>+                invalidateCachedPosition();
</ins><span class="cx">                 GST_DEBUG_OBJECT(pipeline(), "[Seek] seeking to %s failed", toString(m_seekTime).utf8().data());
</span><span class="cx">             }
</span><span class="cx">         }
</span><span class="lines">@@ -2533,7 +2542,7 @@
</span><span class="cx"> 
</span><span class="cx"> void MediaPlayerPrivateGStreamer::didEnd()
</span><span class="cx"> {
</span><del>-    m_cachedPosition = MediaTime::invalidTime();
</del><ins>+    invalidateCachedPosition();
</ins><span class="cx">     MediaTime now = currentMediaTime();
</span><span class="cx">     GST_INFO_OBJECT(pipeline(), "Playback ended, currentMediaTime = %s, duration = %s", now.toString().utf8().data(), durationMediaTime().toString().utf8().data());
</span><span class="cx">     m_isEndReached = true;
</span><span class="lines">@@ -2843,6 +2852,31 @@
</span><span class="cx">     m_canRenderingBeAccelerated = m_player && m_player->acceleratedCompositingEnabled();
</span><span class="cx"> }
</span><span class="cx"> 
</span><ins>+bool MediaPlayerPrivateGStreamer::performTaskAtMediaTime(Function<void()>&& task, const MediaTime& time)
+{
+    ASSERT(isMainThread());
+
+    // Ignore the cases when the time isn't marching on or the position is unknown.
+    MediaTime currentTime = playbackPosition();
+    if (!m_pipeline || m_didErrorOccur || m_isSeeking || m_isPaused || !m_playbackRate || !currentTime.isValid())
+        return false;
+
+    Optional<Function<void()>> taskToSchedule;
+    {
+        auto taskAtMediaTimeScheduler = holdLock(m_TaskAtMediaTimeSchedulerDataMutex);
+        taskAtMediaTimeScheduler->setTask(WTFMove(task), time,
+            m_playbackRate >= 0 ? TaskAtMediaTimeScheduler::Forward : TaskAtMediaTimeScheduler::Backward);
+        taskToSchedule = taskAtMediaTimeScheduler->checkTaskForScheduling(currentTime);
+    }
+
+    // Dispatch the task if the time is already reached. Dispatching instead of directly running the
+    // task prevents infinite recursion in case the task calls performTaskAtMediaTime() internally.
+    if (taskToSchedule.hasValue())
+        RunLoop::main().dispatch(WTFMove(taskToSchedule.value()));
+
+    return true;
+}
+
</ins><span class="cx"> #if USE(TEXTURE_MAPPER_GL)
</span><span class="cx"> PlatformLayer* MediaPlayerPrivateGStreamer::platformLayer() const
</span><span class="cx"> {
</span><span class="lines">@@ -3068,10 +3102,33 @@
</span><span class="cx">     m_player->sizeChanged();
</span><span class="cx"> }
</span><span class="cx"> 
</span><ins>+void MediaPlayerPrivateGStreamer::invalidateCachedPosition() const
+{
+    m_cachedPosition.reset();
+}
+
+void MediaPlayerPrivateGStreamer::invalidateCachedPositionOnNextIteration() const
+{
+    RunLoop::main().dispatch([weakThis = makeWeakPtr(*this), this] {
+        if (!weakThis)
+            return;
+        invalidateCachedPosition();
+    });
+}
+
</ins><span class="cx"> void MediaPlayerPrivateGStreamer::triggerRepaint(GstSample* sample)
</span><span class="cx"> {
</span><span class="cx">     ASSERT(!isMainThread());
</span><span class="cx"> 
</span><ins>+    GstBuffer* buffer = gst_sample_get_buffer(sample);
+    if (buffer && GST_BUFFER_PTS_IS_VALID(buffer)) {
+        // Heuristic to avoid asking for playbackPosition() from a non-main thread.
+        MediaTime currentTime = MediaTime(gst_segment_to_stream_time(gst_sample_get_segment(sample), GST_FORMAT_TIME, GST_BUFFER_PTS(buffer)), GST_SECOND);
+        auto task = holdLock(m_TaskAtMediaTimeSchedulerDataMutex)->checkTaskForScheduling(currentTime);
+        if (task.hasValue())
+            RunLoop::main().dispatch(WTFMove(task.value()));
+    }
+
</ins><span class="cx">     bool shouldTriggerResize;
</span><span class="cx">     {
</span><span class="cx">         Locker sampleLocker { m_sampleMutex };
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformgraphicsgstreamerMediaPlayerPrivateGStreamerh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h (278003 => 278004)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h   2021-05-25 08:33:30 UTC (rev 278003)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h      2021-05-25 12:52:03 UTC (rev 278004)
</span><span class="lines">@@ -39,6 +39,7 @@
</span><span class="cx"> #include <gst/pbutils/install-plugins.h>
</span><span class="cx"> #include <wtf/Atomics.h>
</span><span class="cx"> #include <wtf/Condition.h>
</span><ins>+#include <wtf/DataMutex.h>
</ins><span class="cx"> #include <wtf/Forward.h>
</span><span class="cx"> #include <wtf/Lock.h>
</span><span class="cx"> #include <wtf/LoggerHelper.h>
</span><span class="lines">@@ -185,6 +186,7 @@
</span><span class="cx"> 
</span><span class="cx">     Optional<VideoPlaybackQualityMetrics> videoPlaybackQualityMetrics() final;
</span><span class="cx">     void acceleratedRenderingStateChanged() final;
</span><ins>+    bool performTaskAtMediaTime(Function<void()>&&, const MediaTime&) override;
</ins><span class="cx"> 
</span><span class="cx"> #if USE(TEXTURE_MAPPER_GL)
</span><span class="cx">     PlatformLayer* platformLayer() const override;
</span><span class="lines">@@ -304,7 +306,7 @@
</span><span class="cx">     void setAudioStreamProperties(GObject*);
</span><span class="cx"> 
</span><span class="cx">     virtual bool doSeek(const MediaTime& position, float rate, GstSeekFlags);
</span><del>-    void invalidateCachedPosition() { m_lastQueryTime.reset(); }
</del><ins>+    void invalidateCachedPosition() const;
</ins><span class="cx"> 
</span><span class="cx">     static void setAudioStreamPropertiesCallback(MediaPlayerPrivateGStreamer*, GObject*);
</span><span class="cx"> 
</span><span class="lines">@@ -324,7 +326,7 @@
</span><span class="cx">     Ref<MainThreadNotifier<MainThreadNotification>> m_notifier;
</span><span class="cx">     MediaPlayer* m_player;
</span><span class="cx">     String m_referrer;
</span><del>-    mutable MediaTime m_cachedPosition;
</del><ins>+    mutable Optional<MediaTime> m_cachedPosition;
</ins><span class="cx">     mutable MediaTime m_cachedDuration;
</span><span class="cx">     bool m_canFallBackToLastFinishedSeekPosition { false };
</span><span class="cx">     bool m_isChangingRate { false };
</span><span class="lines">@@ -389,6 +391,33 @@
</span><span class="cx">     Optional<GstVideoDecoderPlatform> m_videoDecoderPlatform;
</span><span class="cx"> 
</span><span class="cx"> private:
</span><ins>+    class TaskAtMediaTimeScheduler {
+    public:
+        enum PlaybackDirection {
+            Forward, // Schedule when targetTime <= currentTime. Used on forward playback, when playbackRate >= 0.
+            Backward // Schedule when targetTime >= currentTime. Used on backward playback, when playbackRate < 0.
+        };
+        void setTask(Function<void()>&& task, const MediaTime& targetTime, PlaybackDirection playbackDirection)
+        {
+            m_targetTime = targetTime;
+            m_task = WTFMove(task);
+            m_playbackDirection = playbackDirection;
+        }
+        Optional<Function<void()>> checkTaskForScheduling(const MediaTime& currentTime)
+        {
+            if (!m_targetTime.isValid() || !currentTime.isFinite()
+                || (m_playbackDirection == Forward && currentTime < m_targetTime)
+                || (m_playbackDirection == Backward && currentTime > m_targetTime))
+                return Optional<Function<void()>>();
+            m_targetTime = MediaTime::invalidTime();
+            return WTFMove(m_task);
+        }
+    private:
+        MediaTime m_targetTime = MediaTime::invalidTime();
+        PlaybackDirection m_playbackDirection = Forward;
+        Function<void()> m_task = Function<void()>();
+    };
+
</ins><span class="cx">     bool isPlayerShuttingDown() const { return m_isPlayerShuttingDown.load(); }
</span><span class="cx">     MediaTime maxTimeLoaded() const;
</span><span class="cx">     void setVideoSourceOrientation(ImageOrientation);
</span><span class="lines">@@ -458,6 +487,7 @@
</span><span class="cx"> #endif
</span><span class="cx"> 
</span><span class="cx">     void configureMediaStreamAudioTracks();
</span><ins>+    void invalidateCachedPositionOnNextIteration() const;
</ins><span class="cx"> 
</span><span class="cx">     Atomic<bool> m_isPlayerShuttingDown;
</span><span class="cx">     GRefPtr<GstElement> m_textSink;
</span><span class="lines">@@ -491,7 +521,6 @@
</span><span class="cx">     mutable unsigned long long m_totalBytes { 0 };
</span><span class="cx">     URL m_url;
</span><span class="cx">     bool m_shouldPreservePitch { false };
</span><del>-    mutable Optional<Seconds> m_lastQueryTime;
</del><span class="cx">     bool m_isLegacyPlaybin;
</span><span class="cx"> #if ENABLE(MEDIA_STREAM)
</span><span class="cx">     RefPtr<MediaStreamPrivate> m_streamPrivate;
</span><span class="lines">@@ -536,6 +565,8 @@
</span><span class="cx">     // This is set to true if no videoflip element has been added to the pipeline.
</span><span class="cx">     bool m_shouldHandleOrientationTags { false };
</span><span class="cx"> 
</span><ins>+    WTF::DataMutex<TaskAtMediaTimeScheduler> m_TaskAtMediaTimeSchedulerDataMutex;
+
</ins><span class="cx"> private:
</span><span class="cx"> #if USE(WPE_VIDEO_PLANE_DISPLAY_DMABUF)
</span><span class="cx">     GUniquePtr<struct wpe_video_plane_display_dmabuf_source> m_wpeVideoPlaneDisplayDmaBuf;
</span></span></pre>
</div>
</div>

</body>
</html>