[webkit-changes] [WebKit/WebKit] 604261: [visionOS] Entering docked mode causes YouTube vid...

[Jean-Yves Avenard]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 60426119f37bc98b065277077dc63276afc3db24
      https://github.com/WebKit/WebKit/commit/60426119f37bc98b065277077dc63276afc3db24
  Author: Jean-Yves Avenard <jya at apple.com>
  Date:   2024-12-06 (Fri, 06 Dec 2024)

  Changed paths:
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm
    M Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm
    M Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.h
    M Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.mm

  Log Message:
  -----------
  [visionOS] Entering docked mode causes YouTube video playback to stall on ToT
https://bugs.webkit.org/show_bug.cgi?id=284160
rdar://140946292

Reviewed by Andy Estes.

When entering docked mode, the video stall because the GPU process crashed following a null deref
in SourceBufferPrivateAVFObjC::stageVideoRenderer. A regression introduced by 287259 at main

Once this issue was fixed, it revealed a lot more issues.

This change is virtually a revert of 287259 at main and a full-redo.

In 287259 at main we considered three possible use cases:
1- We used an AVSampleBufferDisplayLayer with its associated VideoMediaSampleRenderer
2- We used an AVSampleBufferVideoRenderer with its associated VideoMediaSampleRenderer.
3- We used a render-less VideoMediaSampleRenderer in the situation where would we otherwise have a WebCoreDecompressionSession (video not in the dom or a canvas/WebGL attached).

To store 3, we used the same VideoMediaSampleRenderer member as in case 2.
This and others hasted optimisation shortcuts broke playback on visionOS.
Unfortunately, we have poor automated coverage for visionOS and we have no
ability to test some features (spatial mode, docking etc)

In this new version, we go back as close to what the MediaPlayerPrivateMediaSourceAVFObjC
was prior 287259 at main.
We keep the same 3 use case above but we no longer separately store the AVFoundation objects.
Instead we only use VideoMediaSampleRenderer and use it as a holder for the original objects.
In the case where we would before have used a WebCoreDecompressionSession but
instead re-use the VideoMediaSampleRenderer containing the AVSampleBufferDisplayLayer
but instead set it without a render.

If the preference to prefer using a decompression session is set; case
number 3 will never be used. Instead we will just use either 1 or 2;
their associated VideoMediaSampleRenderer will be using a decompression session
internally as needed.

We can continue to use this VideoMediaSampleRenderer as if it had a layer, but
it will instead use internally a decompression session to keep track of the
decoded video frames.

The member name could do with a rename, they currently add to confusion.
However, to ease the transition and code review; I've kept them the same for now.

Manually tested on visionOS. Covered by existing tests on other platforms.

* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setPageIsVisible):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::updateDisplayLayer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureLayer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyLayer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureVideoRenderer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyVideoRenderer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::isUsingRenderlessMediaSampleRenderer const):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureRenderlessVideoMediaSampleRenderer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyRenderlessVideoMediaSampleRenderer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyLayerOrVideoRenderer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::configureLayerOrVideoRenderer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setNeedsPlaceholderImage):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setShouldDisableHDR):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setShouldMaintainAspectRatio):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::updateSpatialTrackingLabel):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::hasVideoRenderer const):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::layerOrVideoRenderer const):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::displayLayer const):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyLayerOrVideoRendererAndCreateRenderlessVideoMediaSampleRenderer): Deleted.
* Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
(WebCore::SourceBufferPrivateAVFObjC::setCDMInstance):
(WebCore::SourceBufferPrivateAVFObjC::enqueueSampleBuffer):
(WebCore::SourceBufferPrivateAVFObjC::configureVideoRenderer):
(WebCore::SourceBufferPrivateAVFObjC::invalidateVideoRenderer):
(WebCore::SourceBufferPrivateAVFObjC::setVideoRenderer):
(WebCore::SourceBufferPrivateAVFObjC::stageVideoRenderer):
* Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.h:
(WebCore::VideoMediaSampleRenderer::as const): Convenience method to easily retrieve the original layer or render it was first created with.

Canonical link: https://commits.webkit.org/287484@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications