<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[204409] trunk</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta">
<dt>Revision</dt> <dd><a href="http://trac.webkit.org/projects/webkit/changeset/204409">204409</a></dd>
<dt>Author</dt> <dd>philn@webkit.org</dd>
<dt>Date</dt> <dd>2016-08-12 03:35:28 -0700 (Fri, 12 Aug 2016)</dd>
</dl>
<h3>Log Message</h3>
<pre>[GStreamer][OWR] Video rendering fixes
https://bugs.webkit.org/show_bug.cgi?id=160764
Reviewed by Xabier Rodriguez-Calvar.
Source/WebCore:
The video renderer of OpenWebRTC now uses OpenGL so we have two
options to correctly handle this in WebKit:
- if USE_GSTREAMER_GL is enabled then we simply pass our GL
appsink to the renderer and there is no need to create glupload
and glcolorconvert element on WebKit side because those elements
are already created on OpenWebRTC side.
- if USE_GSTREAMER_GL is disabled then we need our sink to download
frames from the GPU so that rendering can be done with the WebKit
video sink in main memory.
* platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp:
(WebCore::MediaPlayerPrivateGStreamerBase::createGLAppSink): Split out GL appsink management
to a separate method that can be used by sub-classes.
(WebCore::MediaPlayerPrivateGStreamerBase::createVideoSinkGL): Use new createGLAppSink method.
* platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h:
* platform/graphics/gstreamer/MediaPlayerPrivateGStreamerOwr.cpp:
(WebCore::MediaPlayerPrivateGStreamerOwr::createVideoSink): handle GStreamer-GL configurations
and pass a different element to the renderer depending on the configuration.
Tools:
* gtk/jhbuild.modules: Bump to current OpenWebRTC master.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunkSourceWebCoreChangeLog">trunk/Source/WebCore/ChangeLog</a></li>
<li><a href="#trunkSourceWebCoreplatformgraphicsgstreamerMediaPlayerPrivateGStreamerBasecpp">trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp</a></li>
<li><a href="#trunkSourceWebCoreplatformgraphicsgstreamerMediaPlayerPrivateGStreamerBaseh">trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h</a></li>
<li><a href="#trunkSourceWebCoreplatformgraphicsgstreamerMediaPlayerPrivateGStreamerOwrcpp">trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerOwr.cpp</a></li>
<li><a href="#trunkToolsChangeLog">trunk/Tools/ChangeLog</a></li>
<li><a href="#trunkToolsgtkjhbuildmodules">trunk/Tools/gtk/jhbuild.modules</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunkSourceWebCoreChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/ChangeLog (204408 => 204409)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/ChangeLog 2016-08-12 09:11:09 UTC (rev 204408)
+++ trunk/Source/WebCore/ChangeLog 2016-08-12 10:35:28 UTC (rev 204409)
</span><span class="lines">@@ -1,3 +1,30 @@
</span><ins>+2016-08-12 Philippe Normand <pnormand@igalia.com>
+
+ [GStreamer][OWR] Video rendering fixes
+ https://bugs.webkit.org/show_bug.cgi?id=160764
+
+ Reviewed by Xabier Rodriguez-Calvar.
+
+ The video renderer of OpenWebRTC now uses OpenGL so we have two
+ options to correctly handle this in WebKit:
+
+ - if USE_GSTREAMER_GL is enabled then we simply pass our GL
+ appsink to the renderer and there is no need to create glupload
+ and glcolorconvert element on WebKit side because those elements
+ are already created on OpenWebRTC side.
+ - if USE_GSTREAMER_GL is disabled then we need our sink to download
+ frames from the GPU so that rendering can be done with the WebKit
+ video sink in main memory.
+
+ * platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp:
+ (WebCore::MediaPlayerPrivateGStreamerBase::createGLAppSink): Split out GL appsink management
+ to a separate method that can be used by sub-classes.
+ (WebCore::MediaPlayerPrivateGStreamerBase::createVideoSinkGL): Use new createGLAppSink method.
+ * platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h:
+ * platform/graphics/gstreamer/MediaPlayerPrivateGStreamerOwr.cpp:
+ (WebCore::MediaPlayerPrivateGStreamerOwr::createVideoSink): handle GStreamer-GL configurations
+ and pass a different element to the renderer depending on the configuration.
+
</ins><span class="cx"> 2016-08-11 Sam Weinig <sam@webkit.org>
</span><span class="cx">
</span><span class="cx"> Support WebIDL unions (Part 1)
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformgraphicsgstreamerMediaPlayerPrivateGStreamerBasecpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp (204408 => 204409)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp 2016-08-12 09:11:09 UTC (rev 204408)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp 2016-08-12 10:35:28 UTC (rev 204409)
</span><span class="lines">@@ -799,6 +799,22 @@
</span><span class="cx"> }
</span><span class="cx">
</span><span class="cx"> #if USE(GSTREAMER_GL)
</span><ins>+GstElement* MediaPlayerPrivateGStreamerBase::createGLAppSink()
+{
+ if (!webkitGstCheckVersion(1, 8, 0))
+ return nullptr;
+
+ GstElement* appsink = gst_element_factory_make("appsink", "webkit-gl-video-sink");
+ if (!appsink)
+ return nullptr;
+
+ g_object_set(appsink, "enable-last-sample", FALSE, "emit-signals", TRUE, "max-buffers", 1, nullptr);
+ g_signal_connect(appsink, "new-sample", G_CALLBACK(newSampleCallback), this);
+ g_signal_connect(appsink, "new-preroll", G_CALLBACK(newPrerollCallback), this);
+
+ return appsink;
+}
+
</ins><span class="cx"> GstElement* MediaPlayerPrivateGStreamerBase::createVideoSinkGL()
</span><span class="cx"> {
</span><span class="cx"> // FIXME: Currently it's not possible to get the video frames and caps using this approach until
</span><span class="lines">@@ -812,8 +828,9 @@
</span><span class="cx"> GstElement* videoSink = gst_bin_new(nullptr);
</span><span class="cx"> GstElement* upload = gst_element_factory_make("glupload", nullptr);
</span><span class="cx"> GstElement* colorconvert = gst_element_factory_make("glcolorconvert", nullptr);
</span><ins>+ GstElement* appsink = createGLAppSink();
</ins><span class="cx">
</span><del>- if (!upload || !colorconvert) {
</del><ins>+ if (!appsink || !upload || !colorconvert) {
</ins><span class="cx"> GST_WARNING("Failed to create GstGL elements");
</span><span class="cx"> gst_object_unref(videoSink);
</span><span class="cx">
</span><span class="lines">@@ -821,12 +838,12 @@
</span><span class="cx"> gst_object_unref(upload);
</span><span class="cx"> if (colorconvert)
</span><span class="cx"> gst_object_unref(colorconvert);
</span><ins>+ if (appsink)
+ gst_object_unref(appsink);
</ins><span class="cx">
</span><span class="cx"> return nullptr;
</span><span class="cx"> }
</span><span class="cx">
</span><del>- GstElement* appsink = gst_element_factory_make("appsink", "webkit-gl-video-sink");
-
</del><span class="cx"> gst_bin_add_many(GST_BIN(videoSink), upload, colorconvert, appsink, nullptr);
</span><span class="cx">
</span><span class="cx"> GRefPtr<GstCaps> caps = adoptGRef(gst_caps_from_string("video/x-raw(" GST_CAPS_FEATURE_MEMORY_GL_MEMORY "), format = (string) { RGBA }"));
</span><span class="lines">@@ -837,12 +854,7 @@
</span><span class="cx"> GRefPtr<GstPad> pad = adoptGRef(gst_element_get_static_pad(upload, "sink"));
</span><span class="cx"> gst_element_add_pad(videoSink, gst_ghost_pad_new("sink", pad.get()));
</span><span class="cx">
</span><del>- g_object_set(appsink, "enable-last-sample", FALSE, "emit-signals", TRUE, "max-buffers", 1, nullptr);
-
- if (result) {
- g_signal_connect(appsink, "new-sample", G_CALLBACK(newSampleCallback), this);
- g_signal_connect(appsink, "new-preroll", G_CALLBACK(newPrerollCallback), this);
- } else {
</del><ins>+ if (!result) {
</ins><span class="cx"> GST_WARNING("Failed to link GstGL elements");
</span><span class="cx"> gst_object_unref(videoSink);
</span><span class="cx"> videoSink = nullptr;
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformgraphicsgstreamerMediaPlayerPrivateGStreamerBaseh"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h (204408 => 204409)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h 2016-08-12 09:11:09 UTC (rev 204408)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h 2016-08-12 10:35:28 UTC (rev 204409)
</span><span class="lines">@@ -131,6 +131,7 @@
</span><span class="cx"> #if USE(GSTREAMER_GL)
</span><span class="cx"> static GstFlowReturn newSampleCallback(GstElement*, MediaPlayerPrivateGStreamerBase*);
</span><span class="cx"> static GstFlowReturn newPrerollCallback(GstElement*, MediaPlayerPrivateGStreamerBase*);
</span><ins>+ GstElement* createGLAppSink();
</ins><span class="cx"> GstElement* createVideoSinkGL();
</span><span class="cx"> #endif
</span><span class="cx">
</span></span></pre></div>
<a id="trunkSourceWebCoreplatformgraphicsgstreamerMediaPlayerPrivateGStreamerOwrcpp"></a>
<div class="modfile"><h4>Modified: trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerOwr.cpp (204408 => 204409)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerOwr.cpp 2016-08-12 09:11:09 UTC (rev 204408)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerOwr.cpp 2016-08-12 10:35:28 UTC (rev 204409)
</span><span class="lines">@@ -330,7 +330,21 @@
</span><span class="cx">
</span><span class="cx"> GstElement* MediaPlayerPrivateGStreamerOwr::createVideoSink()
</span><span class="cx"> {
</span><del>- GstElement* sink = MediaPlayerPrivateGStreamerBase::createVideoSink();
</del><ins>+#if USE(GSTREAMER_GL)
+ // No need to create glupload and glcolorconvert here because they are
+ // already created by the video renderer.
+ GstElement* sink = MediaPlayerPrivateGStreamerBase::createGLAppSink();
+ m_videoSink = sink;
+#else
+ GstElement* sink = gst_bin_new(nullptr);
+ GstElement* gldownload = gst_element_factory_make("gldownload", nullptr);
+ GstElement* videoconvert = gst_element_factory_make("videoconvert", nullptr);
+ GstElement* webkitSink = MediaPlayerPrivateGStreamerBase::createVideoSink();
+ gst_bin_add_many(GST_BIN(sink), gldownload, videoconvert, webkitSink, nullptr);
+ gst_element_link_many(gldownload, videoconvert, webkitSink, nullptr);
+ GRefPtr<GstPad> pad = gst_element_get_static_pad(gldownload, "sink");
+ gst_element_add_pad(sink, gst_ghost_pad_new("sink", pad.get()));
+#endif
</ins><span class="cx"> m_videoRenderer = adoptGRef(owr_gst_video_renderer_new(sink));
</span><span class="cx">
</span><span class="cx"> // FIXME: Remove hardcoded video dimensions when the rendering performance:
</span></span></pre></div>
<a id="trunkToolsChangeLog"></a>
<div class="modfile"><h4>Modified: trunk/Tools/ChangeLog (204408 => 204409)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/ChangeLog 2016-08-12 09:11:09 UTC (rev 204408)
+++ trunk/Tools/ChangeLog 2016-08-12 10:35:28 UTC (rev 204409)
</span><span class="lines">@@ -1,3 +1,12 @@
</span><ins>+2016-08-12 Philippe Normand <pnormand@igalia.com>
+
+ [GStreamer][OWR] Video rendering fixes
+ https://bugs.webkit.org/show_bug.cgi?id=160764
+
+ Reviewed by Xabier Rodriguez-Calvar.
+
+ * gtk/jhbuild.modules: Bump to current OpenWebRTC master.
+
</ins><span class="cx"> 2016-08-12 Per Arne Vollan <pvollan@apple.com>
</span><span class="cx">
</span><span class="cx"> Add missing return statement in convertFast function added in r204376.
</span></span></pre></div>
<a id="trunkToolsgtkjhbuildmodules"></a>
<div class="modfile"><h4>Modified: trunk/Tools/gtk/jhbuild.modules (204408 => 204409)</h4>
<pre class="diff"><span>
<span class="info">--- trunk/Tools/gtk/jhbuild.modules 2016-08-12 09:11:09 UTC (rev 204408)
+++ trunk/Tools/gtk/jhbuild.modules 2016-08-12 10:35:28 UTC (rev 204409)
</span><span class="lines">@@ -514,7 +514,7 @@
</span><span class="cx"> <dep package="gst-plugins-openwebrtc"/>
</span><span class="cx"> <dep package="libnice"/>
</span><span class="cx"> </dependencies>
</span><del>- <branch repo="github.com" module="EricssonResearch/openwebrtc.git" checkoutdir="openwebrtc" tag="f511ea1fa79a33fa3d52bfd1c0969c28084aeb35"/>
</del><ins>+ <branch repo="github.com" module="EricssonResearch/openwebrtc.git" checkoutdir="openwebrtc" tag="9b18c94a1c5e5cad16f241954613c267cb032c85"/>
</ins><span class="cx"> </autotools>
</span><span class="cx">
</span><span class="cx"> <autotools id="llvm"
</span></span></pre>
</div>
</div>
</body>
</html>