[webkit-changes] [WebKit/WebKit] 3cf847: [WebCodecs] Poor video frame rate exporting from C...

[Kimmo Kinnunen]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 3cf847f4c0b488559358f6ecdae56a59632266f8
      https://github.com/WebKit/WebKit/commit/3cf847f4c0b488559358f6ecdae56a59632266f8
  Author: Kimmo Kinnunen <kkinnunen at apple.com>
  Date:   2023-10-25 (Wed, 25 Oct 2023)

  Changed paths:
    A LayoutTests/http/wpt/webcodecs/videoFrame-webglCanvasImageSource-expected.txt
    A LayoutTests/http/wpt/webcodecs/videoFrame-webglCanvasImageSource.html
    M Source/WebCore/Modules/mediastream/CanvasCaptureMediaStreamTrack.cpp
    M Source/WebCore/Modules/webcodecs/WebCodecsVideoFrame.cpp
    M Source/WebCore/html/HTMLCanvasElement.cpp
    M Source/WebCore/html/HTMLCanvasElement.h
    M Source/WebCore/html/OffscreenCanvas.cpp
    M Source/WebCore/html/canvas/WebGLRenderingContextBase.cpp
    M Source/WebCore/html/canvas/WebGLRenderingContextBase.h
    M Source/WebCore/platform/graphics/GraphicsContextGL.h
    M Source/WebCore/platform/graphics/angle/GraphicsContextGLANGLE.cpp
    M Source/WebCore/platform/graphics/angle/GraphicsContextGLANGLE.h
    M Source/WebCore/platform/graphics/cocoa/GraphicsContextGLCocoa.h
    M Source/WebCore/platform/graphics/cocoa/GraphicsContextGLCocoa.mm
    M Source/WebCore/platform/graphics/gbm/GraphicsContextGLGBM.cpp
    M Source/WebCore/platform/graphics/gbm/GraphicsContextGLGBM.h
    M Source/WebCore/platform/graphics/texmap/GraphicsContextGLTextureMapperANGLE.cpp
    M Source/WebCore/platform/graphics/texmap/GraphicsContextGLTextureMapperANGLE.h
    M Source/WebKit/GPUProcess/graphics/RemoteGraphicsContextGL.cpp
    M Source/WebKit/GPUProcess/graphics/RemoteGraphicsContextGL.h
    M Source/WebKit/GPUProcess/graphics/RemoteGraphicsContextGL.messages.in
    M Source/WebKit/GPUProcess/graphics/RemoteGraphicsContextGLFunctionsGenerated.h
    M Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextGLProxy.cpp
    M Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextGLProxy.h
    M Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextGLProxyFunctionsGenerated.cpp
    M Tools/Scripts/generate-gpup-webgl

  Log Message:
  -----------
  [WebCodecs] Poor video frame rate exporting from Construct Animate
https://bugs.webkit.org/show_bug.cgi?id=255869
rdar://108459224

Reviewed by Dan Glastonbury.

WebCodecsVideoFrame would use HTMLCanvasElement::toVideoFrame().
Before WebCodecsVideoFrame, this function was intended to be used from
CanvasCaptureMediaStreamTrack, which would capture the composited frame
at the time of capture. Thus WebCodecsVideoFrame would read the
composited frame, not the drawing buffer.

Add functionality to read the WebGL drawing buffer.

Rename GraphicsContextGL::paintRenderingResultsToCanvas() to more
descriptive and consistent name drawSurfaceBufferToImageBuffer.

The added test will fail for premultipliedAlpha == false case. This
will be fixed in later commits.

* LayoutTests/http/wpt/webcodecs/videoFrame-webglCanvasImageSource.html: Added.
* Source/WebCore/Modules/mediastream/CanvasCaptureMediaStreamTrack.cpp:
(WebCore::CanvasCaptureMediaStreamTrack::Source::captureCanvas):
* Source/WebCore/Modules/webcodecs/WebCodecsVideoFrame.cpp:
(WebCore::WebCodecsVideoFrame::create):
* Source/WebCore/html/HTMLCanvasElement.cpp:
(WebCore::HTMLCanvasElement::getImageData):
(WebCore::HTMLCanvasElement::toVideoFrame):
* Source/WebCore/html/HTMLCanvasElement.h:
* Source/WebCore/html/OffscreenCanvas.cpp:
(WebCore::OffscreenCanvas::transferToImageBitmap):
* Source/WebCore/html/canvas/WebGLRenderingContextBase.cpp:
(WebCore::WebGLRenderingContextBase::paintRenderingResultsToCanvas):
(WebCore::WebGLRenderingContextBase::drawingBufferToPixelBuffer):
(WebCore::WebGLRenderingContextBase::surfaceBufferToVideoFrame):
(WebCore::WebGLRenderingContextBase::paintRenderingResultsToPixelBuffer): Deleted.
(WebCore::WebGLRenderingContextBase::paintCompositedResultsToVideoFrame): Deleted.
* Source/WebCore/html/canvas/WebGLRenderingContextBase.h:
* Source/WebCore/platform/graphics/GraphicsContextGL.h:
* Source/WebCore/platform/graphics/angle/GraphicsContextGLANGLE.cpp:
(WebCore::GraphicsContextGLANGLE::drawSurfaceBufferToImageBuffer):
(WebCore::GraphicsContextGLANGLE::withBufferAsNativeImage):
(WebCore::GraphicsContextGLANGLE::drawingBufferToPixelBuffer):
(WebCore::GraphicsContextGLANGLE::paintRenderingResultsToCanvas): Deleted.
(WebCore::GraphicsContextGLANGLE::paintCompositedResultsToCanvas): Deleted.
(WebCore::GraphicsContextGLANGLE::withDrawingBufferAsNativeImage): Deleted.
(WebCore::GraphicsContextGLANGLE::withDisplayBufferAsNativeImage): Deleted.
(WebCore::GraphicsContextGLANGLE::paintRenderingResultsToPixelBuffer): Deleted.
* Source/WebCore/platform/graphics/angle/GraphicsContextGLANGLE.h:
* Source/WebCore/platform/graphics/cocoa/GraphicsContextGLCocoa.h:
* Source/WebCore/platform/graphics/cocoa/GraphicsContextGLCocoa.mm:
(WebCore::GraphicsContextGLCocoa::surfaceBuffer):
(WebCore::GraphicsContextGLCocoa::createPbufferAndAttachIOSurface):
(WebCore::GraphicsContextGLCocoa::surfaceBufferToVideoFrame):
(WebCore::GraphicsContextGLCocoa::withBufferAsNativeImage):
(WebCore::GraphicsContextGLCocoa::displayBuffer): Deleted.
(WebCore::GraphicsContextGLCocoa::drawingBuffer): Deleted.
(WebCore::GraphicsContextGLCocoa::paintCompositedResultsToVideoFrame): Deleted.
(WebCore::GraphicsContextGLCocoa::withDrawingBufferAsNativeImage): Deleted.
(WebCore::GraphicsContextGLCocoa::withDisplayBufferAsNativeImage): Deleted.
* Source/WebCore/platform/graphics/gbm/GraphicsContextGLGBM.cpp:
(WebCore::GraphicsContextGLGBM::surfaceBufferToVideoFrame):
(WebCore::GraphicsContextGLGBM::paintCompositedResultsToVideoFrame): Deleted.
* Source/WebCore/platform/graphics/gbm/GraphicsContextGLGBM.h:
* Source/WebCore/platform/graphics/texmap/GraphicsContextGLTextureMapperANGLE.cpp:
(WebCore::GraphicsContextGLTextureMapperANGLE::surfaceBufferToVideoFrame):
(WebCore::GraphicsContextGLTextureMapperANGLE::paintCompositedResultsToVideoFrame): Deleted.
* Source/WebCore/platform/graphics/texmap/GraphicsContextGLTextureMapperANGLE.h:
* Source/WebKit/GPUProcess/graphics/RemoteGraphicsContextGL.cpp:
(WebKit::RemoteGraphicsContextGL::drawSurfaceBufferToImageBuffer):
(WebKit::RemoteGraphicsContextGL::surfaceBufferToVideoFrame):
(WebKit::RemoteGraphicsContextGL::paintRenderingResultsToCanvas): Deleted.
(WebKit::RemoteGraphicsContextGL::paintCompositedResultsToCanvas): Deleted.
(WebKit::RemoteGraphicsContextGL::paintCompositedResultsToVideoFrame): Deleted.
* Source/WebKit/GPUProcess/graphics/RemoteGraphicsContextGL.h:
* Source/WebKit/GPUProcess/graphics/RemoteGraphicsContextGL.messages.in:
* Source/WebKit/GPUProcess/graphics/RemoteGraphicsContextGLFunctionsGenerated.h:
(drawingBufferToPixelBuffer):
(paintRenderingResultsToPixelBuffer): Deleted.
* Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextGLProxy.cpp:
(WebKit::RemoteGraphicsContextGLProxy::drawSurfaceBufferToImageBuffer):
(WebKit::RemoteGraphicsContextGLProxy::surfaceBufferToVideoFrame):
(WebKit::RemoteGraphicsContextGLProxy::paintRenderingResultsToCanvas): Deleted.
(WebKit::RemoteGraphicsContextGLProxy::paintCompositedResultsToCanvas): Deleted.
(WebKit::RemoteGraphicsContextGLProxy::paintCompositedResultsToVideoFrame): Deleted.
* Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextGLProxy.h:
* Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextGLProxyFunctionsGenerated.cpp:
(WebKit::RemoteGraphicsContextGLProxy::drawingBufferToPixelBuffer):
(WebKit::RemoteGraphicsContextGLProxy::paintRenderingResultsToPixelBuffer): Deleted.
* Tools/Scripts/generate-gpup-webgl:

Canonical link: https://commits.webkit.org/269757@main