[webkit-qt] QtWebKit, the multimedia support today and tomorrow.

Mon Jun 27 04:57:11 PDT 2011

I support this change! :-)

Kenneth

On Fri, Jun 24, 2011 at 11:08 PM, Alexis Menard
<alexis.menard at openbossa.org> wrote:
> Hello,
>
> DISCLAIMER : It may be controversial for some people.
>
> For many months, the QtWebKit team has worked on the multimedia
> support coming with HTML5. We had a first Phonon backend, abandoned
> and really far from feature completion and far from working well. When
> QtMultimedia appeared a new backend was added to the tree providing a
> more extensive API than Phonon and supporting more platforms
> (Symbian). This is the default today if you have Qt Mobility
> installed. Unfortunately Qt Multimedia really lacks of QA and
> stability, the quality of the backends is really heterogenous and
> backends behaves differently. I tried hard to communicate with the Qt
> Multimedia team, they helped a lot but even today Qt Multimedia is not
> good enough and I feel the support is built on unstable foundations.
>
> On the other hand last year Simon and me investigated on the
> multimedia support in WebKit2, he took over when I moved to INdT but
> the result can be seen on the N9 Browser. This implementation is not
> using Qt Multimedia but instead a modified version of the GStreamer
> backend sitting in webkit.org and maintained + used mainly by the GTK
> port. The lesson learned from this implementation is that it was hard
> for us (we don't have particular knowledge of GStreamer) to go into
> the code and hacked it the way we wanted it. But with time we did it
> and at the end we are providing a really good and tailored
> implementation, by supporting texture streaming for example, things we
> couldn't have done in Qt Multimedia easily because we were tight to
> the API and their release schedule. While I was in INdT, I worked on
> the GStreamer backend upstream to make sure it was working well on
> regular linux desktop. It does now and has the same features as
> QtMultimedia. You can enable it by passing DEFINES+=USE_GSTREAMER=1 to
> your build-webkit script. At the end the code is fairly small and
> doesn't have the same complexity as Qt Multimedia GStreamer backend,
> at the end we just playback sounds and videos :D.
>
> After this successful effort I investigated other platforms like Mac.
> Again I looked at what Apple is shipping and I used it. If you pass
> DEFINES+=USE_QTKIT=1 when building QtWebKit on Mac you can enable the
> media support using the QuickTime backend the mac port is using. The
> implementation is working fairly well too and actually support more
> stuff than QtMultimedia does : the fullscreen support is working
> better (animated, Tor-Arne will love it) and has playback controls in
> fullscreen using the existing native theming. I found it pretty easy
> to use and It didn't require that much changes in Apple's code to make
> it work.
>
> So the advantages are :
>
> - We have pretty well working implementation on Mac and Linux (and N9
> by extension).
> - We supports more features than QtMultimedia does.
> - We can hack the webkit.org backend to shape them the way we want.
> - The backends are pretty well tested and covered by layout tests.
> - We got improvements for free.
> - It makes us closer to other ports by talking to them (both Apple and
> GTK teams were *very* nice to help me to implement everything).
> - We have a very low dependency on Qt4, both GStreamer and QTKIT
> backend are not using much (QImage, a top level QWidget from what I
> remember) stuff that will anyway be part of Qt5 (you need to be able
> to render into something, and you need a top level window).
> - We don't need to wait QtMultimedia to work on Qt5 to support media
> elements on QtWebKit next gen.
> - We have hard time to make the media layout tests to run reliably on
> the Qt Linux bot, using the GStreamer backend of webkit make us almost
> the same as with GTK (so we can have almost the same configuration).
>
> Now the drawback of using the backends of webkit.org :
>
> - Makes harder to maintain : we have multiple backends and code path
> (but in the other hand we are not alone to maintain the backends).
> - We don't have today Windows support, WebKit don't have native
> Windows backend (unless we use the QuickTime backend on Windows if
> QuickTime is installed). I would like to implement one based on MS
> APIs.
> - Symbian is missing too. Not sure it is relevant for the WebKit2
> activities as well as Qt5.
> - We don't have Mac bots on the build.webkit.org, making hard for
> Apple to check that they didn't break Qt if we use their backend.
>
> At the end I still think that using the upstream backend is more
> beneficial and some of the disadvantages can be solved by having a Qt
> Mac Bot on build.webkit.org and implementing the missing backend for
> example.
>
> So here is my proposal (for trunk) :
>
> - Removing Phonon backend from the tree. I asked on
> http://lists.kde.org/?l=kde-multimedia&m=130779689626212&w=2 and got
> no answer, which to me means nobody wants to take care of it. It
> brings confusion in the webkit community.
> - Make GStreamer and QuickTime webkit.org backend the default on Mac
> and Linux. I could investigate of using the feature of WebKit to have
> multiple player so that if the webkit.org player can't work it could
> fallback on QtMultimedia.
> - Use QtMultimedia on Windows and Symbian (up until Symbian disappear
> and I give a try to the MS backend). In the meantime again I could
> investigate on Windows to use the QuickTime backend by default and
> fallback to QtMultimedia if QuickTime is not installed.
>
> I think this makes us ready for working on a WebKit2/Qt5, helps Nokia
> but keep the desktop world functional too.
>
> Any thoughts? feedback? Please please answer :D.
>
> PS (running all the media layout tests on my mac) :
>
> QTKIT :
> 117 test cases (80%) succeeded
> 13 test cases (8%) had incorrect layout
> 16 test cases (10%) were new
> 7 test cases (4%) had stderr output
>
> QTMULTIMEDIA :
> 79 test cases (54%) succeeded
> 51 test cases (34%) had incorrect layout
> 16 test cases (10%) were new
> 29 test cases (19%) had stderr output
>
> On Linux :
>
> GSTREAMER :
> 109 test cases (74%) succeeded
> 19 test cases (13%) had incorrect layout
> 16 test cases (10%) were new
> 2 test cases (1%) timed out
> 9 test cases (6%) had stderr output
>
> QTMULTIMEDIA :
> 88 test cases (60%) succeeded
> 41 test cases (28%) had incorrect layout
> 16 test cases (10%) were new
> 1 test case (<1%) timed out
> 19 test cases (13%) had stderr output
>
> Some can be fixed by having pixel tests support but the results are
> much more encouraging with the webkit.org backends.
>
> --
> Alexis Menard
> Software Engineer
> INdT Recife Brazil
> _______________________________________________
> webkit-qt mailing list
> webkit-qt at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-qt
>

-- 
Kenneth Rohde Christiansen
Senior Engineer
Application and Service Frameworks, Nokia Danmark A/S
Phone  +45 4093 0598 / E-mail kenneth.christiansen at gmail.com

http://codeposts.blogspot.com ﹆﹆﹆