[webkit-changes] [WebKit/WebKit] 0e0d79: WebRTC backed media element rendering is slow to u...

[youennf]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 0e0d7976dfe09316e3cb57a4786defeb5c5d7eb6
      https://github.com/WebKit/WebKit/commit/0e0d7976dfe09316e3cb57a4786defeb5c5d7eb6
  Author: Youenn Fablet <youennf at gmail.com>
  Date:   2023-05-31 (Wed, 31 May 2023)

  Changed paths:
    M Source/WebCore/platform/graphics/avfoundation/objc/LocalSampleBufferDisplayLayer.h
    M Source/WebCore/platform/graphics/avfoundation/objc/LocalSampleBufferDisplayLayer.mm
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm

  Log Message:
  -----------
  WebRTC backed media element rendering is slow to update to changes of layers (size or layer made hidden/visible)
https://bugs.webkit.org/show_bug.cgi?id=256695
rdar://109257177

Reviewed by Eric Carlson.

AVSampleBufferDisplayLayer may sometimes take a very long time to update when its bounds are changed.
To workaround this, we are now recreating a new AVSampleBufferDisplayLayer whenever there is a resizing.
We are then enqueuing a VideoFrame if there is one on the current layer.
And we hop back to main thread to replace the old AVSampleBufferDisplayLayer with the new one.

Since we are changing of AVSampleBufferDisplayLayer, we are storing a member on main thread and another on the work queue where we enqueue video frames.
We are also storing the last enqueued pixel buffer in case we need to reenqueue it in the new AVSampleBufferDisplayLayer.
We are adding thread checks for these members.

On MediaPlayerPrivateMediaStreamAVFObjC, we are reducing the number of resizing we were doing.
We remove some class to updateRootLayerBoundsAndPosition that were needed for in process rendering.
We are also delaying creation of the display layer until the size is not empty.

Manually tested.

* Source/WebCore/platform/graphics/avfoundation/objc/LocalSampleBufferDisplayLayer.h:
* Source/WebCore/platform/graphics/avfoundation/objc/LocalSampleBufferDisplayLayer.mm:
(WebCore::LocalSampleBufferDisplayLayer::initialize):
(WebCore::LocalSampleBufferDisplayLayer::~LocalSampleBufferDisplayLayer):
(WebCore::LocalSampleBufferDisplayLayer::updateRootLayerBoundsAndPosition):
(WebCore::LocalSampleBufferDisplayLayer::flush):
(WebCore::LocalSampleBufferDisplayLayer::flushAndRemoveImage):
(WebCore::LocalSampleBufferDisplayLayer::enqueueVideoFrame):
(WebCore::LocalSampleBufferDisplayLayer::enqueueBuffer):
(WebCore::LocalSampleBufferDisplayLayer::removeOldVideoFramesFromPendingQueue):
(WebCore::LocalSampleBufferDisplayLayer::addVideoFrameToPendingQueue):
(WebCore::LocalSampleBufferDisplayLayer::clearVideoFrames):
(WebCore::LocalSampleBufferDisplayLayer::requestNotificationWhenReadyForVideoData):
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::enqueueVideoFrame):
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::ensureLayers):
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::setPresentationSize):
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::layersAreInitialized):
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::characteristicsChanged):
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::setVideoInlineSizeFenced):

Canonical link: https://commits.webkit.org/264777@main