[webkit-gtk] WebKit2GTK+ and compositing in the UIProcess

[zan at falconsigh.net]

On Tue, Dec 8, 2015, at 04:49 PM, Emanuele Aina wrote:
> The current Accelerated Compositing infrastructure in WebKit2GTK+
> relies on the WebProcess painting directly to a X Window owned by the
> UIProcess, which is suboptimal due to the lack of integration with the
> UIProcess paint loop (no synchronization between the WebView contents
> and other widgets and no ability to stack other widgets on top of the
> WebView). Ideally we should do what the Mac port does and export each
> layer from the WebProcess(es) and let the UIProcess composite them in
> its paint loop. GdkGLContext is explicitly meant to solve both the
> synchronization and stacking issues.
> 
> To do that, my current understanding is that the most sensible way
> forward would be to rely on GdkGLContext. in the UIProcess to
> efficiently compose the layers, and make the UIProcess a Wayland
> (sub)compositor to be able to use the Wayland EGL integration (even
> under X, just like Weston).
> 
> Differently from Žan's WebkitForWayland approach where he just need to
> export the handle for the final surface buffer to the WebProcess which
> would then use it as a GL render target, we need to be able to use the
> layer buffers as GL render targets in the WebProcess and as textures in
> the UIProcess. 
> 

The rendering target is actually set up in the WebProcess, simply by
creating a gbm_surface object and using that as the native window for
the EGLSurface.[1]

After the rendering and the buffer swap call, the front buffer is locked
and the
information about the returned gbm_bo object is passed to the UIProcess.
There the buffer object is imported and used as appropriate for the
backend
that's been chosen.[2][3]

> Exporting textures from the UIProcess as dmabufs and use them as render
> targets in the WebProcess is not guaranteed to work in a efficient
> manner, as the GL implementation may want to reallocate the storage
> when it wants to, e.g. to win parallelism, but export would prevent
> that.
> 

I guess this would still apply even with the previous clarification. Not
familiar
with this though, so I honestly can't give you a definitive answer
whether the
wk4wl approach falters here.

> Also using dmabufs directly means that we should have really low-level
> per-platform knowledge as they are created with driver-specific ioclts.

I don't agree. The GBM API abstracts this well enough.

> The general advice from I got is to try to look at higher layer
> approaches, and the most widespread abstraction over the mechanisms we
> need is the EGL_WL_bind_wayland_display extension [3], thus the need to
> be a Wayland (sub)compositor. This does not mean we should implement
> every Wayland feature, we only need to use the protocol to pass buffers
> around leaving input management to the WebKit2 infrastructure. The
> output-only demo nested compositor[1] in the Weston sources is around
> ~1k lines, whitespaces included. We could use a reduced, output-only
> version of the wakefield compositor experiment[2] from Alex Larsson and
> Jasper St. Pierre for this purpose.
> 
> The sequence is more or less this:
> 
>  * UIProcess: calls eglGetDisplay() on the native display (Wayland or
>    X11)
>  * UIProcess: calls eglBindWaylandDisplayWL() to initialize EGL Wayland
>    client support in the Wayland (sub)compositor
>  * WebProcess: calls wl_display_connect() → wl_registry_bind()
>  * WebProcess: calls wl_compositor_create_surface() →
>    wl_egl_window_create() → eglCreateWindowSurface() →
>    cairo_gl_surface_create_for_egl()  to create surfaces for each layer
>    (or just use GL on the EGLSurface)
>  * WebProcess: uses a custom Wayland extension to synchronize the
>    Wayland vs. custom event streams when sending the
>    CoordinatedGraphicsState to the UIProcess
>  * UIProcess: calls eglCreateImageKHR() on the wl_buffers received from
>    the WebProcess
>  * UIProcess: uses the received CoordinatedGraphicsState to composite
>    the EGLImages using GL or even passing them with no copies to the
>    upstream compositor with eglCreateWaylandBufferFromImageWL() [4],
>    but this would probably require the GTK+ scene graph work to land to
>    properly detect if other widgets overlap the WebView
> 

This is how the nested compositor implementation worked before, even if
the work was never fully finished. Compared to using the GBM platform
for rendering in the WebProcess, this approach is proven to work for the
GTK+ port. Whether it works efficiently is debatable (it certainly
wasn't
efficient a few years back).

Skimming the eglCreateWaylandBufferFromImageWL implementation in Mesa,
it appears this would use a dma-buf buffer wherever available anyway.
[4]

> If GL is not available, I guess everything can be done by keeping the
> layers in SHM buffers, and the final compositing happens in the
> UIProcess using plain Cairo. Similar to the
> eglCreateWaylandBufferFromImageWL() case, wl_shm (as a replacement for
> XShm) could be used to upload some layers (eg. video frames) and avoid
> one copy. The same cannot be done under X since Wayland uses POSIX
> shared memory and X11 uses SysV shared memory.
> 
> Žan, Yoon, you're the one who probably have more experience on this,
> would that make sense for you?
> 
> Do you see any issue with the described scenario or do you believe
> there are more simple way to accomplish this?
> 

As I outlined above, personally I am not that supportive of the nested
compositor
approach. But beyond choosing the best way to implement this, the bigger
problem is likely how to use the composition results with the GTK+
toolkit.

Cheers,
Zan

[1]
https://github.com/WebKitForWayland/webkit/blob/master/Source/WPE/Source/Graphics/GBM/RenderingBackendGBM.cpp
[2]
https://github.com/WebKitForWayland/webkit/blob/master/Source/WPE/Source/ViewBackend/DRM/ViewBackendDRM.cpp
[3]
https://github.com/WebKitForWayland/webkit/blob/master/Source/WPE/Source/ViewBackend/Wayland/ViewBackendWayland.cpp
[4]
http://cgit.freedesktop.org/mesa/mesa/tree/src/egl/drivers/dri2/platform_wayland.c#n765

> 
> [1] A stand-alone, output-only reference nested compositor:
>     http://cgit.freedesktop.org/wayland/weston/tree/clients/nested.c
> [2] A proof of concept of a GTK+ Wayland compositor for various 
>     situations: https://github.com/alexlarsson/wakefield
> [3]
> http://cgit.freedesktop.org/mesa/mesa/tree/docs/specs/WL_bind_wayland_display.spec
> [4]
> http://cgit.freedesktop.org/mesa/mesa/tree/docs/specs/WL_create_wayland_buffer_from_image.spec
> 
> -- 
> Emanuele Aina
> www.collabora.com
> 
> 
> 
> _______________________________________________
> webkit-gtk mailing list
> webkit-gtk at lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-gtk