[webkit-changes] [WebKit/WebKit] 38b139: Bounds are not always correctly set when setting t...

[youennf]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 38b13961316ce40f4b73db47c5c97fba86459c3b
      https://github.com/WebKit/WebKit/commit/38b13961316ce40f4b73db47c5c97fba86459c3b
  Author: Youenn Fablet <youennf at gmail.com>
  Date:   2023-07-03 (Mon, 03 Jul 2023)

  Changed paths:
    A LayoutTests/fast/mediastream/video-srcObject-set-twice-expected.html
    A LayoutTests/fast/mediastream/video-srcObject-set-twice.html
    M Source/WebCore/platform/graphics/avfoundation/SampleBufferDisplayLayer.h
    M Source/WebCore/platform/graphics/avfoundation/objc/LocalSampleBufferDisplayLayer.h
    M Source/WebCore/platform/graphics/avfoundation/objc/LocalSampleBufferDisplayLayer.mm
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm
    M Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp
    M Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.h
    M Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.messages.in
    M Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.cpp
    M Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.h

  Log Message:
  -----------
  Bounds are not always correctly set when setting twice a MediaStream to a HTMLMediaElement
https://bugs.webkit.org/show_bug.cgi?id=258723
rdar://110753228

Reviewed by Jer Noble.

After double layer hosting, we were relying on MediaPlayerPrivateMediaStreamAVFObjC::setVideoInlineSizeFenced to be called
so that the AVSampleBufferDisplayLayer bound would be set.

This works fine initially but not when resetting srcObject to a new MediaStream.
In that case, we create a new SampleBufferDisplayLayer but the bounds may not be correctly initialized.
We are making sure to initialize both root and AVSampleBufferDisplayLayer bounds and positions.

They might still not be correctly set due to rotation.
This patch is thus refactoring the code so that LocalSampleBufferDisplayLayer will handle rotation itself.
This is feasible since it receives VideoFrames.
We remove sending affine transforms updates from WebProcess to GPUProcess.
Instead, whenever we receive a VideoFrame in LocalSampleBufferDisplayLayer, we compute the affine transform to use.
If there is a need for a new affine transform, we update the bounds of the AVSampleBufferDisplayLayer, which triggers
creating a new one currently.
This simplifies things as now, only the AVSampleBufferDisplayLayer has to get an affine transform and switch width and height depending on frame rotation.

To reduce flicker, we introduce m_isReconfiguring which will ensure we do not enqueue new video frames to an AVSampleBufferDisplayLayer
that will be removed soon.

We are also using inline video size instead of presentation size if possible to reduce reflowing.
There are still some undesired flicker/reflowing happening when rotation changes since both the media element bounds may change and the rotation.
And these steps are not done at once.

* LayoutTests/fast/mediastream/video-srcObject-set-twice-expected.html: Added.
* LayoutTests/fast/mediastream/video-srcObject-set-twice.html: Added.
* Source/WebCore/platform/graphics/avfoundation/SampleBufferDisplayLayer.h:
* Source/WebCore/platform/graphics/avfoundation/objc/LocalSampleBufferDisplayLayer.h:
* Source/WebCore/platform/graphics/avfoundation/objc/LocalSampleBufferDisplayLayer.mm:
(WebCore::LocalSampleBufferDisplayLayer::initialize):
(WebCore::LocalSampleBufferDisplayLayer::updateBoundsAndPosition):
(WebCore::LocalSampleBufferDisplayLayer::updateSampleLayerBoundsAndPosition):
(WebCore::videoTransformationMatrix):
(WebCore::LocalSampleBufferDisplayLayer::enqueueVideoFrame):
(WebCore::LocalSampleBufferDisplayLayer::enqueueBufferInternal):
(WebCore::LocalSampleBufferDisplayLayer::requestNotificationWhenReadyForVideoData):
(WebCore::LocalSampleBufferDisplayLayer::updateRootLayerAffineTransform): Deleted.
(WebCore::LocalSampleBufferDisplayLayer::updateAffineTransform): Deleted.
(WebCore::LocalSampleBufferDisplayLayer::setRootLayerBoundsAndPositions): Deleted.
(WebCore::LocalSampleBufferDisplayLayer::updateRootLayerBoundsAndPosition): Deleted.
(WebCore::LocalSampleBufferDisplayLayer::enqueueBuffer): Deleted.
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::enqueueVideoFrame):
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::layersAreInitialized):
(WebCore::videoTransformationMatrix):
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::rootLayerBoundsDidChange):
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::setVideoInlineSizeFenced):
* Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp:
(WebKit::RemoteSampleBufferDisplayLayer::updateBoundsAndPosition):
(WebKit::RemoteSampleBufferDisplayLayer::enqueueVideoFrame):
(WebKit::RemoteSampleBufferDisplayLayer::updateAffineTransform): Deleted.
* Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.h:
* Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.messages.in:
* Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.cpp:
(WebKit::SampleBufferDisplayLayer::updateBoundsAndPosition):
(WebKit::SampleBufferDisplayLayer::updateAffineTransform): Deleted.
* Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.h:

Canonical link: https://commits.webkit.org/265708@main