[webkit-qt] QtWebKit, the multimedia support today and tomorrow.

Wed Jun 29 03:07:48 PDT 2011

On Monday, June 27, 2011 01:57:11 pm ext Kenneth Rohde Christiansen wrote:
> I support this change! :-)

Me, too. Even though the situation isn't ideal, it is clearly an improvement from
the what-works-out-of-the-box-and-what-doesn't point of view.

Excellent work has been done and it's time to ship it - it's as simple as that.

Simon

> On Fri, Jun 24, 2011 at 11:08 PM, Alexis Menard
> <alexis.menard at openbossa.org> wrote:
> > Hello,
> >
> > DISCLAIMER : It may be controversial for some people.
> >
> > For many months, the QtWebKit team has worked on the multimedia
> > support coming with HTML5. We had a first Phonon backend, abandoned
> > and really far from feature completion and far from working well. When
> > QtMultimedia appeared a new backend was added to the tree providing a
> > more extensive API than Phonon and supporting more platforms
> > (Symbian). This is the default today if you have Qt Mobility
> > installed. Unfortunately Qt Multimedia really lacks of QA and
> > stability, the quality of the backends is really heterogenous and
> > backends behaves differently. I tried hard to communicate with the Qt
> > Multimedia team, they helped a lot but even today Qt Multimedia is not
> > good enough and I feel the support is built on unstable foundations.
> >
> > On the other hand last year Simon and me investigated on the
> > multimedia support in WebKit2, he took over when I moved to INdT but
> > the result can be seen on the N9 Browser. This implementation is not
> > using Qt Multimedia but instead a modified version of the GStreamer
> > backend sitting in webkit.org and maintained + used mainly by the GTK
> > port. The lesson learned from this implementation is that it was hard
> > for us (we don't have particular knowledge of GStreamer) to go into
> > the code and hacked it the way we wanted it. But with time we did it
> > and at the end we are providing a really good and tailored
> > implementation, by supporting texture streaming for example, things we
> > couldn't have done in Qt Multimedia easily because we were tight to
> > the API and their release schedule. While I was in INdT, I worked on
> > the GStreamer backend upstream to make sure it was working well on
> > regular linux desktop. It does now and has the same features as
> > QtMultimedia. You can enable it by passing DEFINES+=USE_GSTREAMER=1 to
> > your build-webkit script. At the end the code is fairly small and
> > doesn't have the same complexity as Qt Multimedia GStreamer backend,
> > at the end we just playback sounds and videos :D.
> >
> > After this successful effort I investigated other platforms like Mac.
> > Again I looked at what Apple is shipping and I used it. If you pass
> > DEFINES+=USE_QTKIT=1 when building QtWebKit on Mac you can enable the
> > media support using the QuickTime backend the mac port is using. The
> > implementation is working fairly well too and actually support more
> > stuff than QtMultimedia does : the fullscreen support is working
> > better (animated, Tor-Arne will love it) and has playback controls in
> > fullscreen using the existing native theming. I found it pretty easy
> > to use and It didn't require that much changes in Apple's code to make
> > it work.
> >
> > So the advantages are :
> >
> > - We have pretty well working implementation on Mac and Linux (and N9
> > by extension).
> > - We supports more features than QtMultimedia does.
> > - We can hack the webkit.org backend to shape them the way we want.
> > - The backends are pretty well tested and covered by layout tests.
> > - We got improvements for free.
> > - It makes us closer to other ports by talking to them (both Apple and
> > GTK teams were *very* nice to help me to implement everything).
> > - We have a very low dependency on Qt4, both GStreamer and QTKIT
> > backend are not using much (QImage, a top level QWidget from what I
> > remember) stuff that will anyway be part of Qt5 (you need to be able
> > to render into something, and you need a top level window).
> > - We don't need to wait QtMultimedia to work on Qt5 to support media
> > elements on QtWebKit next gen.
> > - We have hard time to make the media layout tests to run reliably on
> > the Qt Linux bot, using the GStreamer backend of webkit make us almost
> > the same as with GTK (so we can have almost the same configuration).
> >
> > Now the drawback of using the backends of webkit.org :
> >
> > - Makes harder to maintain : we have multiple backends and code path
> > (but in the other hand we are not alone to maintain the backends).
> > - We don't have today Windows support, WebKit don't have native
> > Windows backend (unless we use the QuickTime backend on Windows if
> > QuickTime is installed). I would like to implement one based on MS
> > APIs.
> > - Symbian is missing too. Not sure it is relevant for the WebKit2
> > activities as well as Qt5.
> > - We don't have Mac bots on the build.webkit.org, making hard for
> > Apple to check that they didn't break Qt if we use their backend.
> >
> > At the end I still think that using the upstream backend is more
> > beneficial and some of the disadvantages can be solved by having a Qt
> > Mac Bot on build.webkit.org and implementing the missing backend for
> > example.
> >
> > So here is my proposal (for trunk) :
> >
> > - Removing Phonon backend from the tree. I asked on
> > http://lists.kde.org/?l=kde-multimedia&m=130779689626212&w=2 and got
> > no answer, which to me means nobody wants to take care of it. It
> > brings confusion in the webkit community.
> > - Make GStreamer and QuickTime webkit.org backend the default on Mac
> > and Linux. I could investigate of using the feature of WebKit to have
> > multiple player so that if the webkit.org player can't work it could
> > fallback on QtMultimedia.
> > - Use QtMultimedia on Windows and Symbian (up until Symbian disappear
> > and I give a try to the MS backend). In the meantime again I could
> > investigate on Windows to use the QuickTime backend by default and
> > fallback to QtMultimedia if QuickTime is not installed.
> >
> > I think this makes us ready for working on a WebKit2/Qt5, helps Nokia
> > but keep the desktop world functional too.
> >
> > Any thoughts? feedback? Please please answer :D.
> >
> > PS (running all the media layout tests on my mac) :
> >
> > QTKIT :
> > 117 test cases (80%) succeeded
> > 13 test cases (8%) had incorrect layout
> > 16 test cases (10%) were new
> > 7 test cases (4%) had stderr output
> >
> > QTMULTIMEDIA :
> > 79 test cases (54%) succeeded
> > 51 test cases (34%) had incorrect layout
> > 16 test cases (10%) were new
> > 29 test cases (19%) had stderr output
> >
> > On Linux :
> >
> > GSTREAMER :
> > 109 test cases (74%) succeeded
> > 19 test cases (13%) had incorrect layout
> > 16 test cases (10%) were new
> > 2 test cases (1%) timed out
> > 9 test cases (6%) had stderr output
> >
> > QTMULTIMEDIA :
> > 88 test cases (60%) succeeded
> > 41 test cases (28%) had incorrect layout
> > 16 test cases (10%) were new
> > 1 test case (<1%) timed out
> > 19 test cases (13%) had stderr output
> >
> > Some can be fixed by having pixel tests support but the results are
> > much more encouraging with the webkit.org backends.
> >
> > --
> > Alexis Menard
> > Software Engineer
> > INdT Recife Brazil
> > _______________________________________________
> > webkit-qt mailing list
> > webkit-qt at lists.webkit.org
> > http://lists.webkit.org/mailman/listinfo.cgi/webkit-qt
> >
> 
> 
> 
> --
> Kenneth Rohde Christiansen
> Senior Engineer
> Application and Service Frameworks, Nokia Danmark A/S
> Phone  +45 4093 0598 / E-mail kenneth.christiansen at gmail.com
> 
> http://codeposts.blogspot.com ﹆﹆﹆
> _______________________________________________
> webkit-qt mailing list
> webkit-qt at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-qt
> 