[Webkit-unassigned] [Bug 217713] New: currentTime reported as NaN shortly after seeking for MSE playback
bugzilla-daemon at webkit.org
bugzilla-daemon at webkit.org
Wed Oct 14 09:41:09 PDT 2020
https://bugs.webkit.org/show_bug.cgi?id=217713
Bug ID: 217713
Summary: currentTime reported as NaN shortly after seeking for
MSE playback
Product: WebKit
Version: Other
Hardware: Unspecified
OS: Unspecified
Status: NEW
Severity: Normal
Priority: P2
Component: Media
Assignee: webkit-unassigned at lists.webkit.org
Reporter: sycy700 at gmail.com
Created attachment 411335
--> https://bugs.webkit.org/attachment.cgi?id=411335&action=review
web page to reproduce the issue
It looks that media element may return currentTime equals to NaN shortly after seeking. The bug was reproduced on an embedded Linux device, but is not normally seen on the desktop version. Webkit GTK 2.30.1 was used to reproduce the issue.
To reproduce on the desktop:
1. Prepare a media sample, in the test it was MP4/AAC LC audio file. This snippet can be used to make it working for MSE: ffmpeg -i input.mp4 -movflags frag_keyframe+empty_moov -movflags frag_keyframe+empty_moov+default_base_moof output.mp4
2. Patch Webkit by adding a sleep before posting an event to handle end of append:
--- AppendPipeline.cpp.orig 2020-10-14 15:44:50.917918291 +0100
+++ AppendPipeline.cpp 2020-10-14 15:42:30.395406822 +0100
@@ -319,6 +319,7 @@
}
GST_TRACE_OBJECT(m_pipeline.get(), "Posting end-of-append task to the main thread");
+ sleep(1);
m_taskQueue.enqueueTask([this]() {
handleEndOfAppend();
});
3. In Javascript, add a media source. To the media source add a sourcebuffer.
4. Append a buffer of audio data to the sourcebuffer.
5. After 10ms, check currentTime, it's 0.0.
6. Seek to 0.0 by setting the currentTime attribute.
7. After 10ms, check currentTime, it's NaN.
8. After 3s, check currentTime, it's equal to the duration value - it was seeked until the end.
An HTML file with Javascript code is appended to this bug report and it can be used to reproduce the issue as described above.
After looking a bit in the code, it looks that the problem is in the HTMLMediaElement::seekTask function:
HTMLMediaElement::seekTask
...
time = seekableRanges->ranges().nearest(time);
The PlatformTimeRanges::nearest function returns invalidTime when there are no ranges buffered:
MediaTime PlatformTimeRanges::nearest(const MediaTime& time) const
...
if (!count)
return MediaTime::invalidTime();
Then, going back to HTMLMediaElement::seekTask, invalidTime is set as the last seek time:
HTMLMediaElement::seekTask
...
m_lastSeekTime = time;
Then, when asking about the current time, invalidTime will be returned, which is later converted to NaN:
MediaTime HTMLMediaElement::currentMediaTime() const
...
if (m_seeking) {
ALWAYS_LOG(LOGIDENTIFIER, "seeking, returning", m_lastSeekTime);
return m_lastSeekTime;
}
It looks that there is a race condition between seeking and handling the AppendPipeline::handleEndOfAppend function. AppendPipeline::handleEndOfAppend updates the ranges after they were appended:
AppendPipeline::handleEndOfAppend
->SourceBufferPrivateGStreamer::didReceiveAllPendingSamples
-->SourceBuffer::sourceBufferPrivateAppendComplete
--->SourceBuffer::updateBufferedFromTrackBuffers // ranges are updated here
When the seek occurs before AppendPipeline::handleEndOfAppend is called, it may set the current time to invalidTime due to lack of ranges.
There is similar code in Chromium. Not sure if the code below is responsible that this issue cannot be reproduced using Chromium, but one difference is that when there are no ranges available, then 0.0 is returned instead of invalidTime for the 'nearest' function.
https://github.com/chromium/chromium/blob/2420378bc7c24eee30c48455f0c54aa4c7390aa2/third_party/blink/renderer/core/html/media/html_media_element.cc#L2095
void HTMLMediaElement::Seek(double time) {
...
time = seekable_ranges.Nearest(time, now);
https://github.com/chromium/chromium/blob/085b61f6a13d59f59bd39b2eb2b8797849d730f4/third_party/blink/renderer/platform/exported/web_time_range.cc#L130
double WebTimeRanges::Nearest(double new_playback_position,
double current_playback_position) const {
double best_match = 0;
double best_delta = std::numeric_limits<double>::infinity();
for (const WebTimeRange& range : *this) {
...
return best_match;
}
Another issue connected with returning invalidTime as the current time, is when injecting endOfStream. The current position will be set to the duration of the media file:
MediaSource::endOfStream
->MediaSource::streamEndedWithError
-->MediaSource::setDurationInternal
--->MediaSourceGStreamer::durationChanged
---->MediaPlayerPrivateGStreamerMSE::durationChanged
----->MediaPlayer::durationChanged
------>HTMLMediaElement::mediaPlayerDurationChanged
void HTMLMediaElement::mediaPlayerDurationChanged()
...
MediaTime now = currentMediaTime();
MediaTime dur = durationMediaTime();
ALWAYS_LOG(LOGIDENTIFIER, "duration = ", dur, ", current time = ", now);
if (now > dur)
seekInternal(dur);
In this case current time (now) is invalidTime and the condition 'now > dur' will be satisfied, hence the position will be set to the duration value.
The change below causes that the current time won't be set as invalidTime:
--- HTMLMediaElement.cpp.orig 2020-10-14 17:01:28.677475426 +0100
+++ HTMLMediaElement.cpp 2020-10-14 17:02:13.788998928 +0100
@@ -2983,7 +2983,10 @@
clearSeeking();
return;
}
- time = seekableRanges->ranges().nearest(time);
+ if(seekableRanges->ranges().length())
+ time = seekableRanges->ranges().nearest(time);
+ else
+ time = MediaTime::zeroTime();
m_sentEndEvent = false;
m_lastSeekTime = time;
--
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20201014/ecfd9c2d/attachment-0001.htm>
More information about the webkit-unassigned
mailing list