[webkit-qt] Properly releasing video resources when using gstreamer

[Alexis Menard]

Can you post it to webkit-dev so other people can see it.

You will get a broader audience (GTK guys, who mainly maintain the
GStreamer stuff and other persons who work on the media stack).

Thanks.

On Tue, Feb 28, 2012 at 11:34 AM,  <david.corvoysier at orange.com> wrote:
> Hi,
>
> I am working on the integration of QtWebKit+gstreamer on an embedded target,
> and struggling with two issues related to the proper management of video
> resources (typically hardware demuxers, decoders and renderers) through the
> WebKit gstreamer wrapper.
>
> My first issue is related to what appears to me as a bug in the gstreamer
> backend: the audio resources cannot be released upon MediaPlayerPrivate
> destruction because there is a dangling gst_ref on the audio-sink.
>
> By adding an explicit deref in the object destructor, the audio resources
> are properly released:
>
>     if (m_source) {
>         gst_object_unref(m_source);
>         m_source = 0;
>     }
>
>     if (m_videoSinkBin) {
>         gst_object_unref(m_videoSinkBin);
>         m_videoSinkBin = 0;
>     }
>
> ++    if (m_webkitAudioSink) {
> ++      gst_object_unref(GST_OBJECT(m_webkitAudioSink));
> ++      m_webkitAudioSink = 0;
> ++    }
>
>     if (m_playBin) {
>         gst_element_set_state(m_playBin, GST_STATE_NULL);
>         gst_object_unref(GST_OBJECT(m_playBin));
>         m_playBin = 0;
>     }
>
> Could anybody more familiar with the Gstreamer backend confirm there is a
> problem here ?
>
> My second issue is related to the MediaPLayer object lifecycle: this object
> is created by the HTMLMediaElement whenever it needs to load a resource, and
> is stored in the m_player member as an OwnPtr.
>
> My understanding is that the MediaPlayer is released only under the
> following circumstances:
> - implicitly when the HTMLMediaElement loads a new resource, and the "old"
> MediaPLayer OwnPtr goes out of scope when m_player is assigned to a new
> value (see HTMLMediaElement::loadNextSourceChild()) .
>
> - implicitly when the HTMLMediaElement is gc'ed, and the MediaPLayer OwnPtr
> goes out of scope.
> - explicitly, when the HTMLMediaLElement is "stopped", and the resource has
> not been fully downloaded (see HTMLMediaElement::userCancelledLoad).
>
> The issue I have occurs during application shutdown, if the browser was
> displaying a page with a video resource that had been fully downloaded.
>
> What happens is that when the webview is destroyed, the document is detached
> but the MediaPlayer is not destroyed because:
>
> - the HTMLMediaElement is stopped but the document has been fully
> downloaded,
> - the application terminates before the HTMLMediaElement gets gc'ed.
> As a result, the hardware video resources are not properly released.
>
> I first tried to clear memory cache (QWebSettings::clearMemoryCaches();),
> then to force GC (qt_drt_garbageCollector_collect) from the main app, but
> with little success ...
>
> Then I tried disabling the "load complete" test in userCancelledLoad:
>
> void HTMLMediaElement::userCancelledLoad()
> {
>     LOG(Media, "HTMLMediaElement::userCancelledLoad");
>
> --    if (m_networkState == NETWORK_EMPTY || m_completelyLoaded)
> ++    if (m_networkState == NETWORK_EMPTY)
>         return;
>
> … and it worked without visible side-effects, but I am not sure why this
> test had been introduced in the first place: maybe to allow the MediaPlayer
> to be reused when one navigates back to a page in the history ?
>
> So, again, can anyone familiar with the MediaPLayer code give me hints
> allowing me to force a MediaPlayer to be destroyed on shutdown ?
>
> David Corvoysier
>
>
> _______________________________________________
> webkit-qt mailing list
> webkit-qt at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-qt
>



-- 
Alexis Menard (darktears)
Software Engineer
INdT Recife Brazil