[Webkit-unassigned] [Bug 200615] FMP4 segments streamed into a MediaSource results in a black video.

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=200615

--- Comment #19 from Andrew Sampson <andrew at rainway.com> ---
(In reply to Jer Noble from comment #17)
> (In reply to Roman R. from comment #16)
> > I also re-checked ISO BMFF spec to see if there is a special mention as for
> > inifinite stream and it does not have any special meaning (for 32-bit
> > values) except that 0xFFFFFFFF is indicating that duration cannot be
> > determined. Safari would treat this as a number and this results in huge
> > duration. 
> 
> Okay, then that may be a bug either in WebKit or our fMP4 parser.

Adding a comment to this, Chrome and Firefox (In reply to Jer Noble from comment #9)
> (In reply to Roman R. from comment #8)
> > I think the problem is the transition to paused state is never followed by
> > resumed playback even though there is apparently a lot of new data added. It
> > might be possible that play() from our end could resolve this problem more
> > or less (it is hard to tell upfront if such play() could have ay side
> > effects, esp. performance related) and perhaps the complaint here is that
> > other browsers are handling this automatic resume pretty smoothly where
> > Safari media element just stays paused.
> 
> I really don’t understand this argument. If you want to stay as close to the
> live edge as possible, why the heck wouldn’t you call play() immediately
> after your appends complete?
> 
> This is the Faustian Bargain that you make when you adopt MSE. You must take
> full responsibility for all networking, including all networking adjacent
> activities, such as predicting when media is sufficiently buffered and the
> network is sufficiently fast to allow media to play back uninterrupted.


So while we _could_ be using WebRTC video, it comes with a lot of caveats that I don't think are worth getting into here. MSE has worked well for us in other browsers (Chrome/Firefox) and in our side-by-side testing, our MSE method results in lower latency than WebRTC video, which is a huge win.

Chromium is using hints from the MP4 stream to determine whether low latency mode should be used by checking if an MP4 file is created in real-time, such as used in live streaming, as the fragment_duration is not known in advance and this (mehd) box may be omitted. Firefox also has an explicit low latency mode, though this does not rely on hints and seems to be the default behavior. 

In both of these browsers our users can startup a game stream and play with ultra-low latency. That being said we did manage to get Safari working in our test bed using your suggestion just now, however, the video latency seems to be pretty high on average. We see it has a buffer around 70-100 MS (where as Chrome and Firefox can be < 23 MS at times.)

Are there any modifications we can make to our byte stream to trigger low-delay rendering/decoding in Safari like we see in Chromium/Firefox? Or rather, any other suggestions you can offer. 

Supporting all web browsers is really important to us not just for the user convenience, but also to ensure we aren't forced to recommend Chrome as the "best way to play." 

To make it easier for you to reproduce this specific use-case I can get you setup with a Rainway account and instance located near you. Is the email listed here the best place to shoot you a note to coordinate? 

LMK if I can answer any other questions or how our team can be most helpful.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20190829/def115db/attachment-0001.html>