[webkit-changes] [WebKit/WebKit] 597104: visionOS: Enable Dynamic Content Scaling for GPU p...

[Tim Horton]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 597104441dc65adb03dbfe939e55869b44d685f5
      https://github.com/WebKit/WebKit/commit/597104441dc65adb03dbfe939e55869b44d685f5
  Author: Tim Horton <thorton at apple.com>
  Date:   2023-12-08 (Fri, 08 Dec 2023)

  Changed paths:
    M Source/WTF/wtf/PlatformEnableCocoa.h
    M Source/WebCore/WebCore.xcodeproj/project.pbxproj
    M Source/WebCore/platform/graphics/BifurcatedGraphicsContext.h
    M Source/WebCore/platform/graphics/ImageBuffer.cpp
    M Source/WebCore/platform/graphics/ImageBuffer.h
    M Source/WebCore/platform/graphics/RenderingMode.cpp
    M Source/WebCore/platform/graphics/RenderingMode.h
    A Source/WebCore/platform/graphics/cocoa/DynamicContentScalingDisplayList.h
    M Source/WebCore/platform/graphics/cocoa/IOSurface.h
    M Source/WebCore/platform/graphics/cocoa/IOSurface.mm
    M Source/WebKit/GPUProcess/GPUConnectionToWebProcess.h
    M Source/WebKit/GPUProcess/graphics/RemoteImageBuffer.cpp
    M Source/WebKit/GPUProcess/graphics/RemoteImageBuffer.h
    M Source/WebKit/GPUProcess/graphics/RemoteImageBuffer.messages.in
    M Source/WebKit/GPUProcess/graphics/RemoteImageBufferSet.cpp
    M Source/WebKit/GPUProcess/graphics/RemoteImageBufferSet.h
    M Source/WebKit/GPUProcess/graphics/RemoteImageBufferSet.messages.in
    M Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.cpp
    M Source/WebKit/Scripts/webkit/messages.py
    M Source/WebKit/Shared/GPUProcessPreferencesForWebProcess.h
    M Source/WebKit/Shared/GPUProcessPreferencesForWebProcess.serialization.in
    A Source/WebKit/Shared/RemoteLayerTree/DynamicContentScalingBifurcatedImageBuffer.h
    A Source/WebKit/Shared/RemoteLayerTree/DynamicContentScalingBifurcatedImageBuffer.mm
    R Source/WebKit/Shared/RemoteLayerTree/DynamicContentScalingDisplayList.cpp
    R Source/WebKit/Shared/RemoteLayerTree/DynamicContentScalingDisplayList.h
    M Source/WebKit/Shared/RemoteLayerTree/DynamicContentScalingImageBufferBackend.h
    M Source/WebKit/Shared/RemoteLayerTree/DynamicContentScalingImageBufferBackend.mm
    M Source/WebKit/Shared/RemoteLayerTree/RemoteLayerBackingStore.mm
    M Source/WebKit/Shared/RemoteLayerTree/RemoteLayerTreePropertyApplier.mm
    M Source/WebKit/Shared/RemoteLayerTree/RemoteLayerWithInProcessRenderingBackingStore.h
    M Source/WebKit/Shared/RemoteLayerTree/RemoteLayerWithInProcessRenderingBackingStore.mm
    M Source/WebKit/Shared/RemoteLayerTree/RemoteLayerWithRemoteRenderingBackingStore.h
    M Source/WebKit/Shared/RemoteLayerTree/RemoteLayerWithRemoteRenderingBackingStore.mm
    M Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in
    M Source/WebKit/Shared/WebImage.cpp
    M Source/WebKit/Shared/mac/MediaFormatReader/MediaSampleByteRange.h
    M Source/WebKit/SourcesCocoa.txt
    M Source/WebKit/UIProcess/API/APIPageConfiguration.cpp
    M Source/WebKit/UIProcess/RemoteLayerTree/cocoa/RemoteLayerTreeLayers.h
    M Source/WebKit/UIProcess/RemoteLayerTree/cocoa/RemoteLayerTreeLayers.mm
    M Source/WebKit/WebKit.xcodeproj/project.pbxproj
    M Source/WebKit/WebProcess/GPU/graphics/ImageBufferBackendHandle.h
    M Source/WebKit/WebProcess/GPU/graphics/ImageBufferBackendHandleSharing.h
    M Source/WebKit/WebProcess/GPU/graphics/RemoteImageBufferProxy.cpp
    M Source/WebKit/WebProcess/GPU/graphics/RemoteImageBufferSetProxy.cpp
    M Source/WebKit/WebProcess/GPU/graphics/RemoteImageBufferSetProxy.h
    M Source/WebKit/WebProcess/GPU/graphics/cocoa/ImageBufferRemoteIOSurfaceBackend.cpp
    M Source/WebKit/WebProcess/GPU/graphics/cocoa/ImageBufferRemoteIOSurfaceBackend.h
    M Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemote.h
    M Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemote.mm
    M Source/WebKit/WebProcess/WebProcess.cpp

  Log Message:
  -----------
  visionOS: Enable Dynamic Content Scaling for GPU process DOM rendering
https://bugs.webkit.org/show_bug.cgi?id=266058
rdar://114834211

Reviewed by Said Abou-Hallawa.

Enable Dynamic Content Scaling in the GPU process, and, as this was the last
thing blocking DOM GPU process enablement on visionOS, turn it on!

The general flow is:

- the Web Content process makes a normal WebCore Display List streaming ImageBuffer,
  streaming drawing commands to the GPU process.

- the GPU process now makes a DynamicContentScalingBifurcatedImageBuffer,
  which contains the BifurcatedGraphicsContext that used to just sit directly
  in RemoteLayerBackingStore, and bifurcates the drawing commands between
  the bitmap backing store and a DynamicContentScalingImageBufferBackend
  that records the long-lived display list.

- when encoding the front buffer's contents, we synchronously grab the display
  list from the GPU process. This isn't ideal, because it means we encode them
  late, and serially, but it matches the prior design; we can make improvements
  subsequently.

* Source/WTF/wtf/PlatformEnableCocoa.h:
Turn on DOM GPUP on visionOS.

* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/platform/graphics/BifurcatedGraphicsContext.h:
Allow BifurcatedGraphicsContext to be allocated on the heap; this was an aspirational
forbiddance (because BifurcatedGraphicsContext holds two GraphicsContext references),
but now we'll just have to depend on people doing the right thing (at least until
painting code gets smart-pointer-ified).

* Source/WebCore/platform/graphics/ImageBuffer.cpp:
(WebCore::ImageBuffer::ImageBuffer):
Pass the ImageBufferCreationContext down to the next level of the ImageBuffer
constructor, so we can use it when making the bifurcated backend.

(WebCore::ImageBuffer::dynamicContentScalingDisplayList):
Add a failable getter for the display list, like we have for IOSurface and Cairo Surfaces.
After much deliberation, we settled on this approach to extract the display list
instead of sending backend handles back to the WP ahead of time; this turns out to be
infinitely simpler.

* Source/WebCore/platform/graphics/ImageBuffer.h:
(WebCore::ImageBuffer::create):
* Source/WebCore/platform/graphics/RenderingMode.cpp:
(WebCore::operator<<):
* Source/WebCore/platform/graphics/RenderingMode.h:
* Source/WebCore/platform/graphics/cocoa/IOSurface.h:
* Source/WebCore/platform/graphics/cocoa/IOSurface.mm:
(WebCore::surfaceNameToNSString):
(WebCore::IOSurface::nameForRenderingPurpose):
Add a RenderingPurpose for "bitmap only" layers, as a way to indicate semi-generically
to the GPU process that the given layer doesn't need a display list.

* Source/WebCore/platform/graphics/cocoa/DynamicContentScalingDisplayList.h: Renamed from Source/WebKit/Shared/RemoteLayerTree/DynamicContentScalingDisplayList.h.
(WebCore::DynamicContentScalingDisplayList::DynamicContentScalingDisplayList):
(WebCore::DynamicContentScalingDisplayList::displayList const):
(WebCore::DynamicContentScalingDisplayList::takeSurfaces):
* Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in:
Move DynamicContentScalingDisplayList to WebCore.
Generate its encoders.

* Source/WebKit/GPUProcess/GPUConnectionToWebProcess.h:
(WebKit::GPUConnectionToWebProcess::isDynamicContentScalingEnabled const):
* Source/WebKit/UIProcess/API/APIPageConfiguration.cpp:
(API::PageConfiguration::preferencesForGPUProcess const):
* Source/WebKit/Shared/GPUProcessPreferencesForWebProcess.h:
* Source/WebKit/Shared/GPUProcessPreferencesForWebProcess.serialization.in:
Plumb the "dynamic content scaling enabled" preference to the GPU process.

* Source/WebKit/GPUProcess/graphics/RemoteImageBuffer.cpp:
(WebKit::RemoteImageBuffer::dynamicContentScalingDisplayList):
* Source/WebKit/GPUProcess/graphics/RemoteImageBuffer.h:
* Source/WebKit/GPUProcess/graphics/RemoteImageBuffer.messages.in:
* Source/WebKit/WebProcess/GPU/graphics/RemoteImageBufferProxy.cpp:
(WebKit::RemoteImageBufferProxy::RemoteImageBufferProxy):
(WebKit::RemoteImageBufferProxy::dynamicContentScalingDisplayList):
* Source/WebKit/WebProcess/GPU/graphics/RemoteImageBufferProxy.h:
Plumb the display list back to the Web Content process when requested.

* Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.cpp:
(WebKit::isSmallLayerBacking):
(WebKit::createImageBufferInternal):
(WebKit::RemoteRenderingBackend::createImageBuffer):
Factor out creation of the primary image buffer, and override the container type
if we need a bifurcated ImageBuffer (if the preference is enabled and the buffer
is for non-bitmap-only layer backing store).

* Source/WebKit/Shared/RemoteLayerTree/DynamicContentScalingBifurcatedImageBuffer.h: Added.
* Source/WebKit/Shared/RemoteLayerTree/DynamicContentScalingBifurcatedImageBuffer.mm: Added.
(WebKit::DynamicContentScalingBifurcatedImageBuffer::DynamicContentScalingBifurcatedImageBuffer):
(WebKit::DynamicContentScalingBifurcatedImageBuffer::context const):
(WebKit::DynamicContentScalingBifurcatedImageBuffer::dynamicContentScalingDisplayList):
* Source/WebKit/Shared/RemoteLayerTree/DynamicContentScalingImageBufferBackend.h:
* Source/WebKit/Shared/RemoteLayerTree/DynamicContentScalingImageBufferBackend.mm:
(WebKit::DynamicContentScalingImageBufferBackend::createBackendHandle const):
Add DynamicContentScalingBifurcatedImageBuffer, which just overrides
context() to create a BifurcatedGraphicsContext and records the display list.

Clear the context after encoding the display list; this replaces the releaseGraphicsContext()
that used to be in swapToValidFrontBuffer. This ensures that we never accumulate display lists.

* Source/WebKit/Shared/RemoteLayerTree/RemoteLayerBackingStore.h:
* Source/WebKit/Shared/RemoteLayerTree/RemoteLayerBackingStore.mm:
(WebKit::RemoteLayerBackingStore::clearBackingStore):
(WebKit::hasValue):
(WebKit::RemoteLayerBackingStore::encode const):
(WebKit::RemoteLayerBackingStore::ensureFrontBuffer):
(WebKit::RemoteLayerBackingStore::paintContents):
(WebKit::RemoteLayerBackingStoreProperties::layerContentsBufferFromBackendHandle):
(WebKit::RemoteLayerBackingStore::drawInContext): Deleted.
* Source/WebKit/Shared/RemoteLayerTree/RemoteLayerWithInProcessRenderingBackingStore.mm:
(WebKit::allocateBufferInternal):
(WebKit::RemoteLayerWithInProcessRenderingBackingStore::allocateBuffer const):
(WebKit::RemoteLayerWithInProcessRenderingBackingStore::swapToValidFrontBuffer):
* Source/WebKit/Shared/RemoteLayerTree/RemoteLayerWithRemoteRenderingBackingStore.mm:
(WebKit::RemoteLayerWithRemoteRenderingBackingStore::allocateBuffer const):
Remove all of the existing support for bifurcation directly in RemoteLayerBackingStore
and RemoteLayerWithInProcessRenderingBackingStore, adopting
DynamicContentScalingBifurcatedImageBuffer for both in-process and GPUP layers.
Plumb the "containsBitmapOnly" bit along in the RenderingPurpose.

* Source/WebKit/Shared/WebImage.cpp:
(WebKit::WebImage::create):
* Source/WebKit/SourcesCocoa.txt:
* Source/WebKit/UIProcess/RemoteLayerTree/cocoa/RemoteLayerTreeLayers.h:
* Source/WebKit/UIProcess/RemoteLayerTree/cocoa/RemoteLayerTreeLayers.mm:
(-[WKCompositingLayer _setWKContents:withDisplayList:replayForTesting:]):
* Source/WebKit/WebKit.xcodeproj/project.pbxproj:
* Source/WebKit/WebProcess/GPU/graphics/ImageBufferBackendHandle.h:
* Source/WebKit/WebProcess/GPU/graphics/ImageBufferBackendHandleSharing.h:
(WebKit::ImageBufferBackendHandleSharing::dynamicContentScalingDisplayList):
(WebKit::ImageBufferBackendHandleSharing::setBackendHandle):
(WebKit::ImageBufferBackendHandleSharing::hasBackendHandle const): Deleted.
Minor adjustments for new names.

* Source/WebKit/WebProcess/GPU/graphics/cocoa/ImageBufferRemoteIOSurfaceBackend.cpp:
(WebKit::ImageBufferRemoteIOSurfaceBackend::hasBackendHandle const): Deleted.
* Source/WebKit/WebProcess/GPU/graphics/cocoa/ImageBufferRemoteIOSurfaceBackend.h:
Remove hasBackendHandle, which is unused.

* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemote.h:
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/PlatformCALayerRemote.mm:
(WebKit::PlatformCALayerRemote::containsBitmapOnly const):
(WebKit::PlatformCALayerRemote::shouldIncludeDisplayListInBackingStore const):
Expose containsBitmapOnly() for use everywhere we compute the RenderingPurpose.

* Source/WebKit/WebProcess/WebProcess.cpp:
(WebKit::WebProcess::shouldUseRemoteRenderingFor):
Remove the default case and explicitly list all values, so you can't get tripped
up by not populating this switch when you add new ones.

Canonical link: https://commits.webkit.org/271770@main