[Webkit-unassigned] [Bug 200949] Media Source Extensions performance during seek

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=200949

--- Comment #15 from Jer Noble <jer.noble at apple.com> ---
(In reply to Dustin Kerstein from comment #13)

> We use video.readyState in our Chrome/Firefox MSE implementations to prevent
> overloading the decoder, but it appears as though Safari doesn't change its
> readyState when seeking, so we can't throttle the seek requests. Do you know
> if this is intended?  

Yes. Chrome drops to HAVE_METADATA always during a seek, even if they're seeking within a buffered range. This is arguably against the spec, since the HTML spec defines HAVE_METADATA as "no media data is available for the immediate current playback position." If you're seeking within the buffered range, there is definitely media data available for the seek destination. It seems that Chrome is using the decoder state as a proxy for media availability. There's some informational notes to that end in the spec, but nothing normative: "Really the only time the difference is relevant is when painting a video element onto a canvas, where it distinguishes the case where something will be drawn (HAVE_CURRENT_DATA or greater) from the case where nothing is drawn (HAVE_METADATA or less)." WebKit will always have something to draw into a canvas, regardless of whether you seek into an unbuffered range, so even this non-normative note doesn't apply here.

> There is a question here though,
> should WebKit be setting m_shouldGenerateTimestamps when the user sets
> Mode=Sequence?

No, the "generate timestamps flag" is set by the call to addSourceBuffer(), not by setting the mode.  See step 6 of the "addSourceBuffer()" algorithm here: <https://w3c.github.io/media-source/#dom-mediasource-addsourcebuffer>.

> > Frankly, the MSE API is not built for what you're trying to achieve, and I
> > think you're barking up the wrong tree here. If the MSE specification
> > explicitly allowed negative playback rates, you could just specify some
> > negative rate and let the media engine do all its optimizations in your
> > favor, including pre-decoding as-of-yet-undisplayed frames, and dropping
> > frames when the decoder can't keep up with the rate you're requesting. But
> > it doesn't, and you're trying to force it to through seeking.
> 
> I've also recently tried using a non-seeking approach with the Mode=Sequence
> + addSourceBuffer('audio/mpeg') workaround. It works for a little while, but
> the playback always ends up stalled, even when trying to detect this state
> and manually setting play() again.

This is something the spec authors are aware of; they would like to add the ability for clients to control the behavior when the currentTime runs up against the end of the buffered range; clients can ask to "play through" gaps and underruns in buffered ranges. See here: <https://github.com/w3c/media-source/issues/160>

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20200502/be651a03/attachment.htm>