[webkit-dev] Canvas performance and memory usage

[David Hyatt]

On Aug 10, 2010, at 2:38 PM, Stephen White wrote:

> On Mon, Aug 9, 2010 at 6:17 PM, David Hyatt <hyatt at apple.com> wrote:
> There are other cases as well where you want a copy.  Patterns are another example. For example you can create a pattern from another canvas, and I don't think it's supposed to be live if that other canvas later changes.  There are examples in SVG as well where patterns are used and I am pretty sure a copy is the desired behavior.
> 
> I don't think it's as simple as saying "image()" should never copy.
> 
> I spent some time thinking about this from a CG perspective, and I couldn't really come up with a good solution.  We use a bitmap context for efficient pixel access for get/putImageData.  Any platform that wants these functions to be efficient has to keep the pixels easily obtainable.  This means, for example, that we can't produce a live image using CGLayers.  Looking at Qt, the only current platform to return a "live" image to canvas for rendering, it looks like they made this tradeoff, i.e., it looks to me like get/putImageData are slower than other platforms.  (The get is having to call toImage() on a pixmap and then has to convert to obtain a good representation of the data for a QImage).  The put has to render the changes into the pixmap.
> 
> I tried renaming the two methods (image() and imageForRendering()) to copiedImage and liveImage, and ended up with about an 80/20 split at the call sites between the two.  It looked really ugly to me, since in nearly every case the "liveImage" was just being used to paint the current contents of the ImageBuffer, i.e., it was just being passed to drawImage.  The copiedImage was typically passed to somebody who needed to retain it for later rendering, e.g., a pattern.
> 
> So to sum up, the call sites want two things:
> 
> (1) To render the current contents of the image buffer efficiently.
> 
> Would it make sense to add a GraphicsContext::drawImageBuffer(ImageBuffer*, ...) call for this use case?  Then the details of how the image is actually extracted (or not) could be kept down in the platform-specific layers.  I don't know if this entirely helps the problem under discussion, but I ran into it in another context and it seemed like it would be useful.

Yeah, I think an even better way of abstracting it might be to make ImageBuffer:drawIntoContext(GraphicsContext*, ...).  I think that would be simpler for people implementing something special.  If we did that, then the image() accessor on ImageBuffer could probably just always be a deep copy (or copy-on-write).

Getting rid of the graphicsContext->drawImage(imageBuffer->image()....) pattern would definitely be good though.

I'm just really curious about the performance of canvas and whether it's better to have slower get/PutImageData or faster rendering otherwise.   It all comes down to how people are using canvas.  I suspect that get/PutImageData are really really popular.

dave
(hyatt at apple.com)


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20100810/9ed22cfb/attachment.html>