[Webkit-unassigned] [Bug 86410] [texmap][GStreamer] Composited Video support

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=86410





--- Comment #47 from Simon Hausmann <hausmann at webkit.org>  2012-10-26 02:58:22 PST ---
(In reply to comment #46)
> I just start to look at webkit, please correct me if there is any misunderstanding.
> here are some concern from me for the patch:
> 
> 1. paint video to graphics layer by drawTexture() is heavy
> I expect (and think it is) that video is the only RenderObject in a graphics layer, even NOT share the same graphics layer with player controls; otherwise we composite each video frame twice. one is in the backing surface of graphics layer, another is for final compositing of each graphics layer.
> compositing includes glClear/drawTexture/swapBuffer, it is really heavy.
> how about the following suggestion:
> set video texture to graphics layer by bindSurface() and skip (or do nothing) in beginPainting/endPainting/drawTexture

That is kind of what is happening. Each graphics layer has a corresponding TextureMapperPlatformLayer. When the texture mapper renders the scene, it asks each platform layer to paint itself (this is a somewhat simplified model). So if you have a html document with a video element and a bunch of controls on top, you should end up with three platform layers:

    (1) The background / regular content is cached in tile textures as part of a TextureMapperTiledBackingStore (which is a sub-class of TextureMapperPlatformLayer). Its paintTextTextureMapper draws the tile textures.
    (2) The video gets its own TextureMapperPlatformLayer, that's what MediaPlayerPrivateGStreamer is and implements. So after painting the tiles of the static content, it's up to the MediaPlayerPrivateGStreamer to draw the video texture.
    (3) Lastly the controls on top are cached in another layer (I think usually also again a TextureMapperTiledBackingStore) and are drawn in one shot.

So for an entire scene there is no double-composition, the video frames are not rendered into an intermediate buffer.

(This is a WK1 TextureMapperGL model, with WK2 video is not implemented yet and might either get an intermediate GraphicsSurface or if the platform supports it perhaps we could also avoid the extra composition).

> I had thought to use cairo gl surface for video before, since sw path of gtk/efl uses cairo. do you think it is possible to create a simple graphics layer with one texture only(without the complexity of TextureMapper)?

I think that's what we do have today (well, not really, but could :)

> 2. There is both ::paint and ::paintToTextureMapper in MediaPlayerPrivateGStreamer, they may conflict
> I don’t know where ::paint inherits from, but when TextureMapperPlatformLayer is used, should we avoid inherit the interface introduce ::paint function.
> should we only use ::paint in sw path and ::paintToTextureMapper in ACCELERATED_COMPOSITING path?

::paint() will still be called when we're in the accelerated compositing code path, but we do an early return from there
if we are, so no extra pixels are painted. I don't remember exactly where it is called from, it might be the population of the root layer? I've seen it get called at least, but the early return should catch it.

> 3. Create an empty video texture (m_texture), then upload video data to this texture introduce data copy
> I’d like create texture from existing native image like: glEGLImageTargetTexture2DOES() from an existing EGLImage
> how could I configure webkit to use EGL/GLES instead of GLX/GL? for Qt port, it may decided inside qt. I cares more for gtk/efl port.

This may be a missing feature in BitmapTextureGL in the texture mapper I would say. But conceptually it sounds like what we should indeed aim for.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.