[webkit-changes] [WebKit/WebKit] 63ca1c: GPUP deadlock shutting down RemoteRenderingBackend...

[Kimmo Kinnunen]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 63ca1cca5ba566ac40e5786711aab59493f6cbb1
      https://github.com/WebKit/WebKit/commit/63ca1cca5ba566ac40e5786711aab59493f6cbb1
  Author: Kimmo Kinnunen <kkinnunen at apple.com>
  Date:   2024-09-13 (Fri, 13 Sep 2024)

  Changed paths:
    M Source/WebCore/html/HTMLVideoElement.cpp
    M Source/WebCore/html/HTMLVideoElement.h
    M Source/WebCore/html/ImageBitmap.cpp
    M Source/WebCore/html/canvas/CanvasRenderingContext2DBase.cpp
    M Source/WebCore/platform/VideoFrame.cpp
    M Source/WebCore/platform/VideoFrame.h
    M Source/WebCore/platform/graphics/BifurcatedGraphicsContext.cpp
    M Source/WebCore/platform/graphics/BifurcatedGraphicsContext.h
    M Source/WebCore/platform/graphics/GraphicsContext.cpp
    M Source/WebCore/platform/graphics/GraphicsContext.h
    M Source/WebCore/platform/graphics/GraphicsContextGL.cpp
    M Source/WebCore/platform/graphics/MediaPlayer.cpp
    M Source/WebCore/platform/graphics/MediaPlayer.h
    M Source/WebCore/platform/graphics/NullGraphicsContext.h
    M Source/WebCore/platform/graphics/cv/VideoFrameCV.mm
    M Source/WebCore/platform/graphics/displaylists/DisplayListItem.h
    M Source/WebCore/platform/graphics/displaylists/DisplayListItems.cpp
    M Source/WebCore/platform/graphics/displaylists/DisplayListItems.h
    M Source/WebCore/platform/graphics/displaylists/DisplayListRecorder.cpp
    M Source/WebCore/platform/graphics/displaylists/DisplayListRecorder.h
    M Source/WebCore/platform/graphics/displaylists/DisplayListRecorderImpl.cpp
    M Source/WebCore/platform/graphics/displaylists/DisplayListRecorderImpl.h
    M Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp
    M Source/WebCore/platform/graphics/gstreamer/VideoFrameGStreamer.cpp
    M Source/WebCore/platform/graphics/nicosia/cairo/NicosiaCairoOperationRecorder.cpp
    M Source/WebCore/platform/graphics/nicosia/cairo/NicosiaCairoOperationRecorder.h
    M Source/WebCore/rendering/RenderVideo.cpp
    M Source/WebKit/GPUProcess/graphics/RemoteDisplayListRecorder.cpp
    M Source/WebKit/GPUProcess/graphics/RemoteDisplayListRecorder.h
    M Source/WebKit/GPUProcess/graphics/RemoteDisplayListRecorder.messages.in
    M Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.h
    M Source/WebKit/Shared/DisplayListArgumentCoders.serialization.in
    M Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in
    M Source/WebKit/WebProcess/GPU/graphics/RemoteDisplayListRecorderProxy.cpp
    M Source/WebKit/WebProcess/GPU/graphics/RemoteDisplayListRecorderProxy.h
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm

  Log Message:
  -----------
  GPUP deadlock shutting down RemoteRenderingBackend while drawing a MediaPlayer
https://bugs.webkit.org/show_bug.cgi?id=243838
rdar://98523368

Reviewed by Youenn Fablet.

GPUP would deadlock in scenario where:
 - RemoteRenderingBackend would be torn down due to, say, connection
   closing. GPUP main thread would wait to complete to shut down RRB
   work queue.
 - RRB would execute RemoteDisplayListRecorder::paintFrameForMedia
   which would try to run a synchronous task from RRB work queue
   in main thread, painting the media player to a ImageBuffer
Since main thread would be waiting, it wouldn't run the
paintFrameForMedia.

The paintFrameForMedia is a command that draws current frame of media player
into the GraphicsContext. This doesn't particularly make sense, as
the "current time" would refer to the original draw time in WCP, but
as implemented, the frame would be snapshot at the actual draw command
execution time.

Instead, do:
 - Obtain reference to current video frame from the media player
 - Draw that video frame

In GPUP, remote media players create the frames in their queue
(main thread). Remote rendering accesses the video frames by their
read references through the video frame object heap.

* Source/WebCore/html/HTMLVideoElement.cpp:
(WebCore::HTMLVideoElement::paint):
Added the plain paint() function to signify paint from render objects.
This probably existed at some point, as a caller to VideoPlayer::paint().
Otherwise snapshots and WKTR captures would make the video element
signal that the video element is needed for 2d context/WebGL, and
force updates, failing media/video-page-visibility-restriction.html.
The added function uses MediaPlayer::paint(), which was dead code
previously.

(WebCore::HTMLVideoElement::paintCurrentFrameInContext):
* Source/WebCore/html/HTMLVideoElement.h:
* Source/WebCore/html/ImageBitmap.cpp:
(WebCore::ImageBitmap::createCompletionHandler):
* Source/WebCore/html/canvas/CanvasRenderingContext2DBase.cpp:
(WebCore::CanvasRenderingContext2DBase::drawImage):
* Source/WebCore/platform/VideoFrame.cpp:
(WebCore::VideoFrame::draw):
(WebCore::VideoFrame::paintInContext): Deleted.
* Source/WebCore/platform/VideoFrame.h:
* Source/WebCore/platform/graphics/BifurcatedGraphicsContext.cpp:
(WebCore::BifurcatedGraphicsContext::drawVideoFrame):
(WebCore::BifurcatedGraphicsContext::paintFrameForMedia): Deleted.
* Source/WebCore/platform/graphics/BifurcatedGraphicsContext.h:
* Source/WebCore/platform/graphics/GraphicsContext.cpp:
(WebCore::GraphicsContext::drawVideoFrame):
(WebCore::GraphicsContext::paintFrameForMedia): Deleted.
(WebCore::GraphicsContext::paintVideoFrame): Deleted.
* Source/WebCore/platform/graphics/GraphicsContext.h:
* Source/WebCore/platform/graphics/GraphicsContextGL.cpp:
(WebCore::GraphicsContextGL::videoFrameToImage):
* Source/WebCore/platform/graphics/NullGraphicsContext.h:
* Source/WebCore/platform/graphics/cv/VideoFrameCV.mm:
(WebCore::VideoFrame::draw):
(WebCore::VideoFrame::paintInContext): Deleted.
* Source/WebCore/platform/graphics/displaylists/DisplayListItem.h:
* Source/WebCore/platform/graphics/displaylists/DisplayListItems.cpp:
(WebCore::DisplayList::PaintFrameForMedia::PaintFrameForMedia): Deleted.
(WebCore::DisplayList::PaintFrameForMedia::apply const): Deleted.
(WebCore::DisplayList::PaintFrameForMedia::dump const): Deleted.
* Source/WebCore/platform/graphics/displaylists/DisplayListItems.h:
(WebCore::DisplayList::PaintFrameForMedia::identifier const): Deleted.
(WebCore::DisplayList::PaintFrameForMedia::destination const): Deleted.
(WebCore::DisplayList::PaintFrameForMedia::isValid const): Deleted.
* Source/WebCore/platform/graphics/displaylists/DisplayListRecorder.cpp:
(WebCore::DisplayList::Recorder::drawVideoFrame):
(WebCore::DisplayList::Recorder::paintFrameForMedia): Deleted.
(WebCore::DisplayList::Recorder::paintVideoFrame): Deleted.
* Source/WebCore/platform/graphics/displaylists/DisplayListRecorder.h:
* Source/WebCore/platform/graphics/displaylists/DisplayListRecorderImpl.cpp:
(WebCore::DisplayList::RecorderImpl::recordDrawVideoFrame):
(WebCore::DisplayList::RecorderImpl::recordPaintFrameForMedia): Deleted.
(WebCore::DisplayList::RecorderImpl::recordPaintVideoFrame): Deleted.
* Source/WebCore/platform/graphics/displaylists/DisplayListRecorderImpl.h:
* Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
(WebCore::MediaPlayerPrivateGStreamer::paint):
* Source/WebCore/platform/graphics/gstreamer/VideoFrameGStreamer.cpp:
(WebCore::VideoFrame::draw):
(WebCore::VideoFrame::paintInContext): Deleted.
* Source/WebCore/platform/graphics/nicosia/cairo/NicosiaCairoOperationRecorder.cpp:
(Nicosia::CairoOperationRecorder::drawVideoFrame):
(Nicosia::CairoOperationRecorder::paintFrameForMedia): Deleted.
* Source/WebCore/platform/graphics/nicosia/cairo/NicosiaCairoOperationRecorder.h:
* Source/WebCore/rendering/RenderVideo.cpp:
(WebCore::RenderVideo::paintReplaced):
* Source/WebKit/GPUProcess/graphics/RemoteDisplayListRecorder.cpp:
(WebKit::RemoteDisplayListRecorder::drawVideoFrame):
(WebKit::RemoteDisplayListRecorder::paintFrameForMedia): Deleted.
(WebKit::RemoteDisplayListRecorder::paintVideoFrame): Deleted.
* Source/WebKit/GPUProcess/graphics/RemoteDisplayListRecorder.h:
* Source/WebKit/GPUProcess/graphics/RemoteDisplayListRecorder.messages.in:
* Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.h:
* Source/WebKit/Shared/DisplayListArgumentCoders.serialization.in:
* Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in:
* Source/WebKit/WebProcess/GPU/graphics/RemoteDisplayListRecorderProxy.cpp:
(WebKit::RemoteDisplayListRecorderProxy::recordDrawVideoFrame):
(WebKit::RemoteDisplayListRecorderProxy::recordPaintFrameForMedia): Deleted.
(WebKit::RemoteDisplayListRecorderProxy::recordPaintVideoFrame): Deleted.
* Source/WebKit/WebProcess/GPU/graphics/RemoteDisplayListRecorderProxy.h:

Canonical link: https://commits.webkit.org/283637@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications