[Webkit-unassigned] [Bug 200615] New: FMP4 segments streamed into a MediaSource results in a black video.

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=200615

            Bug ID: 200615
           Summary: FMP4 segments streamed into a MediaSource results in a
                    black video.
           Product: WebKit
           Version: Safari 12
          Hardware: Macintosh
                OS: macOS 10.14
            Status: NEW
          Severity: Major
          Priority: P2
         Component: Media
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: andrew at rainway.com

Hi, we operate a game streaming service that enables end-users to play PC games inside of their web browser via a low-latency network stream. Safari in the past worked (albeit without Opus support) however, since we changed the format of our video to better support low-latency decoding, Safari fails to render anything when passing data to a MediaSource via append buffer. 

You can download a dump of one of our streams from https://images.rainway.com/videos/demos/StreamDump.mp4.zip to analyze.

Some information on our setup:

We stream real-time video by appending MSE data chunks and our goal is to have lowest latency decoding/presentation of this data. In most cases we are interested in having zero scheduling on client side (by browser) since if there is a video frame data it is already time to show it without bothering about time stamps and scheduling for presentation what would normal video delivery assume. Still we have to format the data to look like traditional fragmented MP4 streams, with frame rate indication in particular or otherwise browsers tend to fail to play the content. Our streams do not contain I-Frames after the first initialization segment. 

We believe our content is okay for low latency overall. Specifically in non-browser scenarios (e.g. in Xbox One MediaElement control which is essentially close in terms of pipeline structure: receives fragments MP4, decodes with the help of DXVA2, schedules for GPU-enabled presentation - like in browsers with just a bit more of control over the pipeline from our end) the same content shows good stable ultra low latency. Our app also works both in Chrome (https://images.rainway.com/videos/demos/Chrome_75_Mac.mp4) and in Firefox (https://images.rainway.com/videos/demos/Firefox_69_Windows.mp4)


We've disabled support for Safari in our web app for now, but would love to restore it as soon as possible. Please let me know if I can provide any other information.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20190810/e165f63c/attachment.html>