[Webkit-unassigned] [Bug 196348] New: [MSE][GStreamer] Incorrect notification of a source buffer about AppSrc being ready to more data

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=196348

            Bug ID: 196348
           Summary: [MSE][GStreamer] Incorrect notification of a source
                    buffer about AppSrc being ready to more data
           Product: WebKit
           Version: WebKit Local Build
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: WebKitGTK
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: s.krutikov at activevideo.com
                CC: bugs-noreply at webkitgtk.org

The bug happens on our proprietary platform, which is based on WebKitGTK. We have added an extra GStreamer pipeline that consumes data from the output of PlayBin pipeline. There is a MpegTsMux element which generates an encoded stream using this data in a synchronous manner. The symptom of the bug is a playback getting stuck due to the lack of video data, which makes a muxer in the extra pipeline to not produce any stream data. We have found this bug playing movies in Hulu application.

The problem is caused by incorrect notification of a video source buffer about video AppSrc in PlayBin pipeline needs more data. The notification is made in the corresponding callback of GstAppSrc element (https://github.com/WebKit/webkit/blob/master/Source/WebCore/platform/graphics/gstreamer/mse/WebKitMediaSourceGStreamer.cpp#L138). However, there is a problem in how this notification is made. In fact, it should be dispatched to the main thread using the corresponding method of the notifier (https://github.com/WebKit/webkit/blob/master/Source/WebCore/platform/graphics/gstreamer/MainThreadNotifier.h#L44). Careful examination of this code reveals a hidden problem: if a notification of a particular type has already been added (i.e. it is pending currently) but has not been handled yet, then a new notification of the same type will be lost. This might happen because both audio and video AppSrc share the same notification type (https://github.com/WebKit/webkit/blob/master/Source/WebCore/platform/graphics/gstreamer/mse/WebKitMediaSourceGStreamerPrivate.h#L99).

If we assume that a buffer in audio AppSink of PlayBin pipeline is quite small, then 'need data' event will happen much more often for audio AppSrc than video AppSrc. This is exactly what sometimes happens in our platform. The consequence is that at some moment after handling a pending notification a new one is added, and it is always for audio since there are far more events for audio than for video. So 'need data' events for video get lost. This leads to 'm_isReadyForMoreSamples' flag staying in False state (https://github.com/WebKit/webkit/blob/master/Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.cpp#L138), which prevents video source buffer from pushing data to PlayBin pipeline (https://github.com/WebKit/webkit/blob/master/Source/WebCore/Modules/mediasource/SourceBuffer.cpp#L2016).

In our platform we have fixed the bug by introducing a special flag to a stream structure (https://github.com/WebKit/webkit/blob/master/Source/WebCore/platform/graphics/gstreamer/mse/WebKitMediaSourceGStreamerPrivate.h#L49). Notification function (https://github.com/WebKit/webkit/blob/master/Source/WebCore/platform/graphics/gstreamer/mse/WebKitMediaSourceGStreamer.cpp#L678) has been changed so that it iterates over all streams and notifies those which have this flag set.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20190328/9fc8665f/attachment.html>