[webkit-changes] [WebKit/WebKit] 897c86: Eliminate double layer hosting of video content wh...

Mon Feb 20 16:12:43 PST 2023

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 897c865996c5bcef29fb5b058df0dc6ae1172466
      https://github.com/WebKit/WebKit/commit/897c865996c5bcef29fb5b058df0dc6ae1172466
  Author: Jer Noble <jer.noble at apple.com>
  Date:   2023-02-20 (Mon, 20 Feb 2023)

  Changed paths:
    M Source/WebCore/Headers.cmake
    M Source/WebCore/PAL/pal/spi/cocoa/AVKitSPI.h
    M Source/WebCore/WebCore.xcodeproj/project.pbxproj
    M Source/WebCore/html/HTMLMediaElement.cpp
    M Source/WebCore/html/HTMLMediaElement.h
    A Source/WebCore/html/HTMLMediaElementIdentifier.h
    M Source/WebCore/platform/cocoa/VideoFullscreenModel.h
    M Source/WebCore/platform/cocoa/VideoFullscreenModelVideoElement.h
    M Source/WebCore/platform/cocoa/VideoFullscreenModelVideoElement.mm
    M Source/WebCore/platform/cocoa/WebAVPlayerLayer.h
    M Source/WebCore/platform/cocoa/WebAVPlayerLayer.mm
    A Source/WebCore/platform/cocoa/WebAVPlayerLayerView.h
    A Source/WebCore/platform/cocoa/WebAVPlayerLayerView.mm
    M Source/WebCore/platform/graphics/GraphicsLayer.h
    M Source/WebCore/platform/graphics/MediaPlayer.cpp
    M Source/WebCore/platform/graphics/MediaPlayer.h
    M Source/WebCore/platform/graphics/MediaPlayerPrivate.h
    M Source/WebCore/platform/graphics/ca/GraphicsLayerCA.cpp
    M Source/WebCore/platform/graphics/ca/GraphicsLayerCA.h
    M Source/WebCore/platform/ios/VideoFullscreenInterfaceAVKit.h
    M Source/WebCore/platform/ios/VideoFullscreenInterfaceAVKit.mm
    M Source/WebCore/platform/mac/VideoFullscreenInterfaceMac.mm
    M Source/WebCore/rendering/RenderLayerBacking.cpp
    M Source/WebKit/Platform/cocoa/LayerHostingContext.h
    M Source/WebKit/Platform/cocoa/LayerHostingContext.mm
    M Source/WebKit/Shared/PlaybackSessionContextIdentifier.h
    M Source/WebKit/Shared/RemoteLayerTree/RemoteLayerTreeTransaction.h
    M Source/WebKit/Shared/RemoteLayerTree/RemoteLayerTreeTransaction.mm
    M Source/WebKit/Shared/WebExtensionContextIdentifier.h
    M Source/WebKit/Shared/WebExtensionControllerIdentifier.h
    M Source/WebKit/UIProcess/Cocoa/VideoFullscreenManagerProxy.h
    M Source/WebKit/UIProcess/Cocoa/VideoFullscreenManagerProxy.mm
    M Source/WebKit/UIProcess/RemoteLayerTree/RemoteLayerTreeHost.h
    M Source/WebKit/UIProcess/RemoteLayerTree/RemoteLayerTreeHost.mm
    M Source/WebKit/UIProcess/RemoteLayerTree/ios/RemoteLayerTreeHostIOS.mm
    M Source/WebKit/UIProcess/RemoteLayerTree/mac/RemoteLayerTreeDrawingAreaProxyMac.h
    M Source/WebKit/UIProcess/RemoteLayerTree/mac/RemoteLayerTreeDrawingAreaProxyMac.mm
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h
    M Source/WebKit/WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm
    M Source/WebKit/WebProcess/WebPage/RemoteLayerTree/GraphicsLayerCARemote.cpp
    M Source/WebKit/WebProcess/WebPage/RemoteLayerTree/GraphicsLayerCARemote.h
    M Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemote.cpp
    M Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemote.h
    M Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemoteCustom.h
    M Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemoteCustom.mm
    M Source/WebKit/WebProcess/WebPage/RemoteLayerTree/RemoteLayerTreeContext.h
    M Source/WebKit/WebProcess/WebPage/RemoteLayerTree/RemoteLayerTreeContext.mm
    M Source/WebKit/WebProcess/WebPage/WebPage.cpp
    M Source/WebKit/WebProcess/WebPage/mac/TiledCoreAnimationDrawingArea.mm
    M Source/WebKit/WebProcess/cocoa/PlaybackSessionManager.h
    M Source/WebKit/WebProcess/cocoa/PlaybackSessionManager.mm
    M Source/WebKit/WebProcess/cocoa/VideoFullscreenManager.h
    M Source/WebKit/WebProcess/cocoa/VideoFullscreenManager.mm

  Log Message:
  -----------
  Eliminate double layer hosting of video content when UI-Side compositing is enabled
https://bugs.webkit.org/show_bug.cgi?id=252021
<rdar://105246033>

Reviewed by Eric Carlson and Per Arne Vollan.

Currently, video content from the GPU process is remotely-hosted into the WebContent
process, and is subsequently remotely hosted from the WebContent process to the UI
process when UI-side compositing is enabled. This is unnecessary, and necessitates
having an active CARenderer in the Web Content process. Rather than double hosting
video content from GPU -> WebContent -> UI process, video content should be hosted
directly from the GPU process into the UI process.

To facilitate this, re-use the infrastructure built into VideoFullscreenManager to
handle the required XPC communication to send resize information back down from the
UI process to the GPU process.

HTMLMediaElement will need to expose API to resize its video layer, and to provide
the remote hosting context ID from the GPU process.

Rather than have a separate identifier of its own, VideoFullscreenManager and
PlaybackSessionManager can just use a HTMLMediaElement identifier for the purposes
of passing messages across the XPC boundary.

WebAVPlayerLayer must support use cases more like VideoLayerRemoteCocoa, including
handling multiple VideoGravity settings, so it is improved to take these settings
into account.

WebAVPlayerLayerView has been pulled out of VideoFullScreenManagerAVKit into its
own source and header files.

GraphicsLayerCA, RemoteLayerTreeHost, and RemoteLayerTree all must be modified to
accept a HTMLVideoElement (or its identifier) as a parameter.

* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/dom/ImageOverlay.cpp:
(WebCore::ImageOverlay::updateSubtree):
* Source/WebCore/html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::setVideoInlineSizeFenced):
(WebCore::HTMLMediaElement::layerHostingContextID):
(WebCore::HTMLMediaElement::naturalSize):
* Source/WebCore/html/HTMLMediaElement.h:
(WebCore::HTMLMediaElement::identifier const):
* Source/WebCore/platform/cocoa/VideoFullscreenModel.h:
* Source/WebCore/platform/cocoa/VideoFullscreenModelVideoElement.h:
* Source/WebCore/platform/cocoa/VideoFullscreenModelVideoElement.mm:
(WebCore::VideoFullscreenModelVideoElement::setVideoInlineSizeFenced):
* Source/WebCore/platform/cocoa/WebAVPlayerLayer.h:
* Source/WebCore/platform/cocoa/WebAVPlayerLayer.mm:
(-[WebAVPlayerLayer layoutSublayers]):
(-[WebAVPlayerLayer resolveBounds]):
* Source/WebCore/platform/graphics/GraphicsLayer.h:
(WebCore::GraphicsLayer::layerMode const):
(WebCore::GraphicsLayer::setContentsToVideoElement):
* Source/WebCore/platform/graphics/MediaPlayer.cpp:
(WebCore::MediaPlayer::setVideoInlineSizeFenced):
(WebCore::MediaPlayer::hostingContextID const):
* Source/WebCore/platform/graphics/MediaPlayer.h:
* Source/WebCore/platform/graphics/MediaPlayerPrivate.h:
(WebCore::MediaPlayerPrivateInterface::setVideoInlineSizeFenced):
(WebCore::MediaPlayerPrivateInterface::hostingContextID const):
* Source/WebCore/platform/graphics/ca/GraphicsLayerCA.cpp:
(WebCore::GraphicsLayerCA::createPlatformCALayer):
(WebCore::GraphicsLayerCA::setContentsToPlatformLayer):
(WebCore::GraphicsLayerCA::setContentsToVideoElement):
* Source/WebCore/platform/graphics/ca/GraphicsLayerCA.h:
* Source/WebCore/platform/graphics/ca/PlatformCALayer.cpp:
(WebCore::operator<<):
* Source/WebCore/platform/graphics/ca/PlatformCALayer.h:
* Source/WebCore/platform/graphics/ca/cocoa/PlatformCALayerCocoa.mm:
(WebCore::PlatformCALayerCocoa::PlatformCALayerCocoa):
* Source/WebCore/platform/graphics/ca/cocoa/WebVideoContainerLayer.mm:
(-[WebVideoContainerLayer setPosition:]): Deleted.
* Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.h:
* Source/WebCore/platform/ios/VideoFullscreenInterfaceAVKit.h:
* Source/WebCore/platform/mac/VideoFullscreenInterfaceMac.h:
* Source/WebCore/platform/mac/VideoFullscreenInterfaceMac.mm:
(-[WebVideoFullscreenInterfaceMacObjC setUpPIPForVideoView:withFrame:inWindow:]):
(WebCore::VideoFullscreenInterfaceMac::createLayer):
* Source/WebCore/rendering/RenderLayerBacking.cpp:
(WebCore::RenderLayerBacking::updateConfiguration):
* Source/WebKit/Platform/cocoa/LayerHostingContext.h:
* Source/WebKit/Platform/cocoa/LayerHostingContext.mm:
(WebKit::LayerHostingContext::createForPort):
(WebKit::LayerHostingContext::createForExternalHostingProcess):
(WebKit::LayerHostingContext::createTransportLayerForRemoteHosting):
(WebKit::LayerHostingContext::updateCachedContextID):
(WebKit::LayerHostingContext::cachedContextID):
* Source/WebKit/Shared/PlaybackSessionContextIdentifier.h:
(): Deleted.
* Source/WebKit/Shared/RemoteLayerTree/RemoteLayerBackingStore.mm:
(WebKit::RemoteLayerBackingStore::drawInContext):
* Source/WebKit/Shared/RemoteLayerTree/RemoteLayerTreeTransaction.h:
* Source/WebKit/Shared/RemoteLayerTree/RemoteLayerTreeTransaction.mm:
(WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::LayerCreationProperties):
(WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::encode const):
(WebKit::RemoteLayerTreeTransaction::LayerCreationProperties::decode):
(WebKit::RemoteLayerTreeTransaction::description const):
* Source/WebKit/UIProcess/Cocoa/VideoFullscreenManagerProxy.h:
* Source/WebKit/UIProcess/Cocoa/VideoFullscreenManagerProxy.mm:
(WebKit::VideoFullscreenManagerProxy::createLayerWithID):
(WebKit::VideoFullscreenManagerProxy::createLayerHostViewWithID):
(WebKit::VideoFullscreenManagerProxy::setupFullscreenWithID):
(WebKit::VideoFullscreenManagerProxy::didCleanupFullscreen):
* Source/WebKit/UIProcess/RemoteLayerTree/RemoteLayerTreeHost.mm:
(WebKit::RemoteLayerTreeHost::makeNode):
* Source/WebKit/UIProcess/RemoteLayerTree/ios/RemoteLayerTreeHostIOS.mm:
(WebKit::RemoteLayerTreeHost::makeNode):
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
(WebKit::MediaPlayerPrivateRemote::hostingContextID const):
(WebKit::MediaPlayerPrivateRemote::setLayerHostingContextID):
(WebKit::MediaPlayerPrivateRemote::setCachedPresentationSize):
(WebKit::MediaPlayerPrivateRemote::cachedPresentationSize):
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
* Source/WebKit/WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm:
(WebKit::MediaPlayerPrivateRemote::layerHostingContextIdChanged):
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/GraphicsLayerCARemote.cpp:
(WebKit::GraphicsLayerCARemote::createPlatformCALayer):
(WebKit::GraphicsLayerCARemote::layerMode const):
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/GraphicsLayerCARemote.h:
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemote.cpp:
(WebKit::PlatformCALayerRemote::create):
(WebKit::PlatformCALayerRemote::recursiveBuildTransaction):
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemote.h:
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemoteCustom.h:
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemoteCustom.mm:
(WebKit::PlatformCALayerRemoteCustom::create):
(WebKit::PlatformCALayerRemoteCustom::PlatformCALayerRemoteCustom):
(WebKit::PlatformCALayerRemoteCustom::hostingContextID):
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/RemoteLayerTreeContext.h:
(WebKit::RemoteLayerTreeContext::webPage):
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/RemoteLayerTreeContext.mm:
(WebKit::RemoteLayerTreeContext::layerDidEnterContext):
* Source/WebKit/WebProcess/WebPage/WebPage.cpp:
(WebKit::m_appHighlightsVisible):
* Source/WebKit/WebProcess/WebPage/mac/TiledCoreAnimationDrawingArea.mm:
(WebKit::TiledCoreAnimationDrawingArea::sendDidFirstLayerFlushIfNeeded):
(WebKit::TiledCoreAnimationDrawingArea::sendEnterAcceleratedCompositingModeIfNeeded):
(WebKit::TiledCoreAnimationDrawingArea::setLayerHostingMode):
* Source/WebKit/WebProcess/cocoa/PlaybackSessionManager.h:
* Source/WebKit/WebProcess/cocoa/PlaybackSessionManager.mm:
(WebKit::PlaybackSessionManager::removeContext):
(WebKit::PlaybackSessionManager::setUpPlaybackControlsManager):
(WebKit::PlaybackSessionManager::contextIdForMediaElement):
* Source/WebKit/WebProcess/cocoa/VideoFullscreenManager.h:
* Source/WebKit/WebProcess/cocoa/VideoFullscreenManager.mm:
(WebKit::VideoFullscreenManager::setupRemoteLayerHosting):
(WebKit::VideoFullscreenManager::enterVideoFullscreenForVideoElement):
(WebKit::VideoFullscreenManager::setVideoLayerFrameFenced):

Co-authored-by: Arunsundar Kannan <arunsundar_kannan at apple.com>

Canonical link: https://commits.webkit.org/260575@main