[webkit-changes] [WebKit/WebKit] da9744: Draws to unused 2D contexts may consume excessive ...

[Kimmo Kinnunen]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: da9744107a306e658c6874a688488457e6cdc47a
      https://github.com/WebKit/WebKit/commit/da9744107a306e658c6874a688488457e6cdc47a
  Author: Kimmo Kinnunen <kkinnunen at apple.com>
  Date:   2024-02-06 (Tue, 06 Feb 2024)

  Changed paths:
    M Source/WebCore/Modules/mediastream/CanvasCaptureMediaStreamTrack.cpp
    M Source/WebCore/Modules/mediastream/CanvasCaptureMediaStreamTrack.h
    M Source/WebCore/Modules/webxr/WebXRWebGLLayer.h
    M Source/WebCore/dom/Document.cpp
    M Source/WebCore/dom/Document.h
    M Source/WebCore/html/CanvasBase.cpp
    M Source/WebCore/html/CanvasBase.h
    M Source/WebCore/html/CanvasObserver.h
    M Source/WebCore/html/HTMLCanvasElement.cpp
    M Source/WebCore/html/HTMLCanvasElement.h
    M Source/WebCore/html/OffscreenCanvas.cpp
    M Source/WebCore/html/canvas/CanvasRenderingContext.h
    M Source/WebCore/html/canvas/CanvasRenderingContext2DBase.cpp
    M Source/WebCore/html/canvas/CanvasRenderingContext2DBase.h
    M Source/WebCore/html/canvas/GPUBasedCanvasRenderingContext.cpp
    M Source/WebCore/html/canvas/GPUBasedCanvasRenderingContext.h
    M Source/WebCore/html/canvas/GPUCanvasContextCocoa.mm
    M Source/WebCore/html/canvas/ImageBitmapRenderingContext.cpp
    M Source/WebCore/html/canvas/WebGLRenderingContextBase.cpp
    M Source/WebCore/inspector/agents/InspectorCanvasAgent.cpp
    M Source/WebCore/inspector/agents/InspectorCanvasAgent.h
    M Source/WebCore/page/Page.cpp
    M Source/WebCore/page/Page.h
    M Source/WebCore/platform/graphics/ImageBuffer.cpp
    M Source/WebCore/platform/graphics/ImageBuffer.h
    M Source/WebCore/rendering/style/StyleCanvasImage.cpp
    M Source/WebCore/rendering/style/StyleCanvasImage.h

  Log Message:
  -----------
  Draws to unused 2D contexts may consume excessive amount of memory on Cocoa
https://bugs.webkit.org/show_bug.cgi?id=268608
rdar://117708049

Reviewed by Simon Fraser.

Accelerated CG applies a drawn operation only once a specific draw
operation limit is hit, if the result is needed as a source to other
draw or on explicit flush. If the operations are not implicitly or
explicitly flushed, the draws are retained in the draw queue.

This causes memory leaks in cases where modifiable surfaces are used as
sources for draws and and then subsequently the the surfaces are
modified. The modifications force copy-on-write for the surfaces that
have pending references in other draw queues.

This can be triggered by drawing a canvas A to a 2D context B
and then never using the result, subsequently modifying A.

Fix by adding a list of flushed CanvasRenderingContexts to Document.
Any 2D context modified during any JS callstack will be put to the
flush list.

Flush the contexts during rendering update, PrepareCanvases phase.
Rename the rendering update PrepareCanvasesForDisplay phase
PrepareCanvases phase, and do both.

PrepareCanvases phase of Document now does two things:
  - flushes deferred operations
  - prepares for display

Currently only Document manages the canvas preparation.
This works for OffscreenCanvas and HTMLCanvasElement in Web main run
loop.

WorkerGlobalContext is not implemented in this patch.
In future, PrepareCanvases phase of WorkerGlobalContext will do
the flush operations, but not the prepares for display. OffscreenCanvas
does not have the prepare for display, as it's not displaying.

* Source/WebCore/Modules/mediastream/CanvasCaptureMediaStreamTrack.cpp:
(WebCore::CanvasCaptureMediaStreamTrack::Source::canvasChanged):
* Source/WebCore/dom/Document.cpp:
(WebCore::Document::prepareCanvasesIfNeeded):
(WebCore::Document::updateCanvasPreparationForDisplayOrFlush):
(WebCore::Document::removeCanvasPreparationForDisplayOrFlush):
(WebCore::Document::prepareCanvasesForDisplayIfNeeded): Deleted.
(WebCore::Document::clearCanvasPreparation): Deleted.
(WebCore::Document::canvasChanged): Deleted.
(WebCore::Document::canvasDestroyed): Deleted.
* Source/WebCore/dom/Document.h:

Remove Document CanvasObserver interface. The interface was making
the actual logic harder than needed and did not abstract anything, as
the clearCanvasPreparation was a non-interface method anyway.

The object which does the prepare (Document, WorkerGlobalContext), is
always statically known at call site. Thus the call site does not need
to jump thorough the canvasChanged hoop.

* Source/WebCore/html/HTMLCanvasElement.cpp:
(WebCore::HTMLCanvasElement::~HTMLCanvasElement):
(WebCore::HTMLCanvasElement::createContext2d):
(WebCore::HTMLCanvasElement::createContextWebGL):
(WebCore::HTMLCanvasElement::createContextWebGPU):
(WebCore::HTMLCanvasElement::didDraw):
(WebCore::HTMLCanvasElement::setSurfaceSize):
(WebCore::HTMLCanvasElement::createImageBuffer const):
(WebCore::HTMLCanvasElement::didMoveToNewDocument):
(WebCore::HTMLCanvasElement::insertedIntoAncestor): Deleted.
(WebCore::HTMLCanvasElement::removedFromAncestor): Deleted.

Remove redundant insertedIntoAncestor, removedFromAncestor handlers.
These are called when the element is still in the same document, but
detached from the tree. This does not cause any changes to the
prepare list, as canvases need to be prepared regardless whether
they're attached or detached.

WebGL still will not prepareForDisplay if it's not in tree, unless
it has been captured by media stream.

* Source/WebCore/html/HTMLCanvasElement.h:
* Source/WebCore/html/OffscreenCanvas.cpp:
(WebCore::OffscreenCanvas::~OffscreenCanvas):
(WebCore::OffscreenCanvas::didDraw):
(WebCore::OffscreenCanvas::updateCanvasPreparation):
(WebCore::OffscreenCanvas::removeCanvasPreparation):
* Source/WebCore/html/OffscreenCanvas.h:
* Source/WebCore/html/canvas/CanvasRenderingContext.h:
(WebCore::CanvasRenderingContext::hasDeferredOperations const):
(WebCore::CanvasRenderingContext::flushDeferredOperations):
* Source/WebCore/html/canvas/CanvasRenderingContext2DBase.cpp:
(WebCore::CanvasRenderingContext2DBase::hasDeferredOperations const):
(WebCore::CanvasRenderingContext2DBase::flushDeferredOperations):
(WebCore::CanvasRenderingContext2DBase::didDraw):
(WebCore::CanvasRenderingContext2DBase::needsPreparationForDisplay const):
* Source/WebCore/html/canvas/CanvasRenderingContext2DBase.h:
* Source/WebCore/page/Page.cpp:
(WebCore::Page::doAfterUpdateRendering):
(WebCore::operator<<):
* Source/WebCore/page/Page.h:
* Source/WebCore/platform/graphics/ImageBuffer.cpp:
(WebCore::ImageBuffer::flushDrawingContextAsync):
* Source/WebCore/platform/graphics/ImageBuffer.h:
(WebCore::ImageBuffer::prefersPreparationForDisplay): Deleted.

Remove unused ImageBuffer::prefersPreparationForDisplay.

Canonical link: https://commits.webkit.org/274164@main