[webkit-dev] WebKit GPU rendering possibility

Thu Nov 3 00:42:32 PDT 2016

El jue, 03-11-2016 a las 07:35 +0000, Rogovin, Kevin escribió:
> Hi,
> 
>  The main issue of making a Cairo backend to FastUIDraw is clipping.
> Cairo tracks the clipping region in CPU and does things that are fine
> for CPU-based rendering (i.e. span based rendering) but are
> absolutely awful for GPU rendering (from my slides, one sees that GL
> backed QPainter and Cairo do much worse than CPU backed). FastUIDraw
> only supports clipIn and clipOut and pushes all the clipping work to
> the GPU with almost no CPU work. It does NOT track the clipping
> region at all. I can give more technical details how it works (and
> those details are why FastUIDraw cannot be used a backend for Cairo).
> For those interested in where the code is located for clipping in
> FastUIDraw, it is located at src/fastuidraw/painter/painter.cpp,
> methods clipInRect, clipOutPath and clipInPath. Their implementations
> are very short and simple and are quite cheap on CPU.

I see. Then I guess adding a new GraphicsContext for FastUIDraw is the
easiest and best way to try this out in WebKit. Would it be possible to
 just add a new GraphicsContext implementation? or would you also need
to change other parts of the graphics implementation or the
GraphicsContext API itself?

> Best Regards,
> -Kevin
> 
> -----Original Message-----
> From: Carlos Garcia Campos [mailto:carlosgc at webkit.org] 
> Sent: Thursday, November 3, 2016 9:27 AM
> To: Rogovin, Kevin <kevin.rogovin at intel.com>; Myles C. Maxfield <mmax
> field at apple.com>
> Cc: webkit-dev at lists.webkit.org
> Subject: Re: [webkit-dev] WebKit GPU rendering possibility
> 
> El jue, 03-11-2016 a las 06:58 +0000, Rogovin, Kevin escribió:
> > Hi!
> >  
> > Question answers:
> > 1.      Currently FastUIDraw has a backend to OpenGL 3.3 and OpenGL
> > ES 
> > 3.0. One of its design goals is to make it not terribly awful to
> > write 
> > a backend to different 3D API’s.
> > 2.      I think I was unclear in my video. I have NOT migrated ANY
> > UI 
> > rendering library to use Fast UI Draw. What I have done is made a
> > demo 
> > (painter-cells) and ported that demo to Fast UI Draw, Cairo, Qt’s 
> > QPainter and SKIA. The diffs between the ports is almost trivial
> > (it 
> > really is just using those different rendering API’s).
> 
> That makes me wonder, would it be possible to add a new cairo backend
> based on FastUIDraw? That would make very easy to try it out with the
> current GraphicsContext cairo backend.
> 
> > 3.      There are a few areas:
> > a.      Reduce some render to offscreen buffers. When I worked
> > with 
> > WebKit YEARS ago, I saw a few instances of rendering to texture
> > that 
> > are unnecessary and even harm performance for GPU rendering. The
> > first 
> > example was where a brush pattern with an image and/or gradient 
> > applied is to be drawn tiled across an area. WebKit (at that time) 
> > first drew a single instance of that pattern to an image, then
> > drew 
> > that image tiled. For GPU renderers we can (very easily) just do
> > the 
> > repeat pattern (of both original image and gradient) from a shader.
> > Another instance happens at RenderLayer where a new
> > GraphicsContext 
> > “layer” is started on a transformation that has rotation or 
> > perspective. For FastUIDraw, this is not necessary, though if a
> > layer 
> > is transparent, then it is.
> > b.      In addition, FastUIDraw has an interface so that if “what”
> > is 
> > drawn is unchanged but the “how” changes, then a caller can cache
> > the 
> > “what” to send to the GPU. To be explicit, “what” to draw is 
> > essentially attributes and indices and those values do NOT depend
> > on 
> > the state of “how” to draw. Examples of “how” to draw: current 
> > transformation, brush to apply, clipping applied, stroking
> > parameters 
> > (including dash pattern) and blending mode. I admit that I am
> > quite 
> > proud of being able to use the same attributes an indices even if 
> > stroking parameters (stroking width, miter limit and dash pattern) 
> > change. Text rendering “what” to draw does depend on what glyphs
> > one 
> > wants to use. Specifically, if drawing coverage font glyphs, then 
> > attributes and indices values change if one wants to draw the
> > glyph 
> > biffer, but for the GPU rendered glyphs they do not.
> > 4.      The renderer implements full 3x3 transformations. However,
> > the 
> > renderer does NOT implement out-of-order transparency. For a GPU,
> > a 
> > 3x3 transformation is cheap (naturally!). The renderer does
> > handle, 
> > with a very little additional overhead changing clipping even
> > between 
> > nasty rotations or perspective changes. The demo painter-cells 
> > deliberately pushes and does lots of nasty clipping and the 
> > performance impact of it on FastUIDraw is very small.
> > 5.      Drawing text is a right pain in the rear. Currently, 
> > FastUIDraw has 3 methods to draw text: coverage, distance field and
> > an 
> > original GPU algorithm that I devised for another open sourced
> > project 
> > years ago. Coverage is needed when glyphs are drawn small and
> > hinting 
> > becomes important. The original GPU algorithm keeps corners sharps
> > and 
> > does a computation in the fragment shader to compute a coverage
> > value. 
> > Distance field is a fall back which has render quality issues
> > (namely 
> > corners are rounded) but is very, very cheap.
> > I want to write an additional glyph renderer that is much faster
> > than 
> > the original GPU method and keeps corners sharp. This new one is
> > to 
> > use the ideas found in https://github.com/Chlumsky/msdfgen but I
> > have 
> > a way to make the distance field generation much, much faster and 
> > handle natively cubics (instead of breaking cubics into quadratics)
> >  
> > For convenience, below is a list of features FastUIDraw implements:
> > 1.      3x3 transformation matrix
> > 2.      path stroking with anti-aliasing a.      dashed stroking
> > too 
> > b.      miter, rouned, bevel joins c.      flat, square and
> > rounded 
> > caps 3.      path filling against an arbitrary fill rule 4.      
> > “brush”
> > a.      linear gradients
> > b.      two point conical gradients (I call these radial gradients)
> > c.      
> > images
> >                                                     i.    
> > nearest, 
> > bilinear and bicubic filtering 5.      Clipping a.      clipIn
> > against 
> > rect or filled path (with arbitrary fill rule) b.      clipOut
> > against 
> > path (with arbitrary fill rule) 6.      Glyph rendering a.      
> > coverage fonts b.      1-channel distance field c.      curve-pair 
> > analytic (original algorithm) 7.      all 12 Porter-Duff blend
> > modes
> >  
> > However, I still have work to do:
> > 1.      anti-alias path fills
> > 2.      anti-alias clipping
> > 3.      more glyph rendering work
> > 4.      some optimizations related to culling on path-fills
> > 5.      
> > dash pattern adjustments from contour length as found in http 
> > s://www.w3.org/TR/svg-strokes/ 6.      the analog of
> > GraphicContext’s 
> > begin/endTransparencyLayer 7.      The blend/combine/transfer modes
> > of 
> > W3C that are not from Porter-Duff.
> >  
> > At this point, I need to garner interest to be able to get time to 
> > work on this project at my employer. The stronger the enthusiasm I
> > can 
> > get the better my chances for continuing the work.
> 
> This looks really interesting, I think the GTK+ port could benefit
> from this if it eventually can be used as a cairo replacement.
>  
> > Best Regards,
> > -Kevin Rogovin
> >  
> > From: Myles C. Maxfield [mailto:mmaxfield at apple.com]
> > Sent: Thursday, November 3, 2016 1:30 AM
> > To: Rogovin, Kevin <kevin.rogovin at intel.com>
> > Cc: webkit-dev at lists.webkit.org
> > Subject: Re: [webkit-dev] WebKit GPU rendering possibility
> >  
> > Hello,
> >  
> > This is certainly interesting work! I have a few questions about
> > the 
> > approach of this renderer.
> >  
> > 1. What API is this on top of? OpenGL? Metal? Vulkan? Raw GPU 
> > commands[1]?
> > 2. You mention in your video that you have already migrated Cairo
> > on 
> > top of your new tech. Traditionally, a web engine is divorced from
> > a 
> > 2D rendering engine such as Cairo. Why can’t the ports of WebKit
> > which 
> > use Cairo get this new tech without any change?
> > 3. What sort of API changes do you have in mind to make 
> > GraphicsContext adopt?
> > 4. Out of curiosity, does the renderer implement 3D transforms?
> > Did 
> > you have to implement 3-D triangle subdivision along intersections 
> > (perhaps for order-independent transparency)?
> > 5. Which algorithm did you choose to draw text?
> >  
> > Historically, the WebKit team has hesitated to allow experiments
> > in 
> > the OpenSource repository. Traditionally, this sort of exploratory 
> > work is done in a branch, and only after it has proved to be an 
> > improvement, the work is adopted on trunk.
> >  
> > Thanks,
> > Myles
> >  
> > [1] https://01.org/linuxgraphics/documentation/hardware-specificati
> > on
> > -prms
> >  
> > On Nov 2, 2016, at 9:35 AM, Rogovin, Kevin <kevin.rogovin at intel.com
> > >
> > wrote:
> >  
> > Hi,
> >  
> > I was directed here by some colleagues as this is the place to
> > post 
> > the following to get started on the following proposal.
> >  
> > I have been working on an experimental 2D renderer that requires a 
> > GPU, the project is open sourced on github at https://github.com/01
> > or 
> > g/fastuidraw. I gave a talk at the X Developers Conference this
> > year 
> > which can be seen from https://www.x.org/wiki/Events/XDC2016/Progra
> > m/
> > rogovin_fast_ui_draw/ .
> >  
> > I made a benchmark which makes heavy use of rotations and clipping
> > and 
> > ported to SKIA, Qt’s QPainter and Cairo. The benchmark and its
> > ports 
> > are in the git repo linked above under the branch 
> > with_ports_of_painter-cells. It's performance advantage of
> > FastUIDraw 
> > against the other renderers was quite severe (against Cairo and
> > Qt's 
> > QPainter over 9 times and against SKIA about 5 times faster).
> >  
> > I would like to explore the option of using FastUIDraw to implement
> > a 
> > WebCore::GraphicsContext backend for the purpose of making drawing 
> > faster and more efficient on Intel devices that are equipped with
> > a 
> > GPU. I also think that some minor modifications to WebKit’s use of 
> > GraphicsContext will also give some benefits. I have worked on
> > WebKit 
> > a few years ago and knew/know my way around the rendering code
> > very 
> > well (atleast at that time).
> >  
> > Looking forward to collaboration,
> > -Kevin Rogovin
> >  
> > _______________________________________________
> > webkit-dev mailing list
> > webkit-dev at lists.webkit.org
> > https://lists.webkit.org/mailman/listinfo/webkit-dev
> >  
> > _______________________________________________
> > webkit-dev mailing list
> > webkit-dev at lists.webkit.org
> > https://lists.webkit.org/mailman/listinfo/webkit-dev