[webkit-qt] Properly releasing video resources when using gstreamer

[david.corvoysier at orange.com]

Hi,

I am working on the integration of QtWebKit+gstreamer on an embedded
target, and struggling with two issues related to the proper management
of video resources (typically hardware demuxers, decoders and renderers)
through the WebKit gstreamer wrapper.

My first issue is related to what appears to me as a bug in the
gstreamer backend: the audio resources cannot be released upon
MediaPlayerPrivate destruction because there is a dangling gst_ref on
the audio-sink.
By adding an explicit deref in the object destructor, the audio
resources are properly released:

    if (m_source) {
        gst_object_unref(m_source);
        m_source = 0;
    }

    if (m_videoSinkBin) {
        gst_object_unref(m_videoSinkBin);
        m_videoSinkBin = 0;
    }

++    if (m_webkitAudioSink) {
++   	gst_object_unref(GST_OBJECT(m_webkitAudioSink));  
++    	m_webkitAudioSink = 0;
++    }

    if (m_playBin) {
        gst_element_set_state(m_playBin, GST_STATE_NULL);
        gst_object_unref(GST_OBJECT(m_playBin));
        m_playBin = 0;
    }   

Could anybody more familiar with the Gstreamer backend confirm there is
a problem here ?

My second issue is related to the MediaPLayer object lifecycle: this
object is created by the HTMLMediaElement whenever it needs to load a
resource, and is stored in the m_player member as an OwnPtr.
My understanding is that the MediaPlayer is released only under the
following circumstances:
- implicitly when the HTMLMediaElement loads a new resource, and the
"old" MediaPLayer OwnPtr goes out of scope when m_player is assigned to
a new value (see HTMLMediaElement::loadNextSourceChild()) .
- implicitly when the HTMLMediaElement is gc'ed, and the MediaPLayer
OwnPtr goes out of scope.
- explicitly, when the HTMLMediaLElement is "stopped", and the resource
has not been fully downloaded (see HTMLMediaElement::userCancelledLoad).

The issue I have occurs during application shutdown, if the browser was
displaying a page with a video resource that had been fully downloaded.
What happens is that when the webview is destroyed, the document is
detached but the MediaPlayer is not destroyed because:
- the HTMLMediaElement is stopped but the document has been fully
downloaded, 
- the application terminates before the HTMLMediaElement gets gc'ed.
As a result, the hardware video resources are not properly released.

I first tried to clear memory cache
(QWebSettings::clearMemoryCaches();), then to force GC
(qt_drt_garbageCollector_collect) from the main app, but with little
success ...

Then I tried disabling the "load complete" test in userCancelledLoad:

void HTMLMediaElement::userCancelledLoad()
{
    LOG(Media, "HTMLMediaElement::userCancelledLoad");

--    if (m_networkState == NETWORK_EMPTY || m_completelyLoaded)
++    if (m_networkState == NETWORK_EMPTY)
        return;
  
... and it worked without visible side-effects, but I am not sure why
this test had been introduced in the first place: maybe to allow the
MediaPlayer to be reused when one navigates back to a page in the
history ?

So, again, can anyone familiar with the MediaPLayer code give me hints
allowing me to force a MediaPlayer to be destroyed on shutdown ?

David Corvoysier   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-qt/attachments/20120228/977416bd/attachment.html>