[webkit-changes] [WebKit/WebKit] 3dd074: Sync wait in RemoteLayerWithRemoteRenderingBacking...

mattwoodrow noreply at github.com
Fri Dec 8 11:20:43 PST 2023 

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 3dd074a9e5d206e613ec48c3d3f57388f00c62c6
      https://github.com/WebKit/WebKit/commit/3dd074a9e5d206e613ec48c3d3f57388f00c62c6
  Author: Matt Woodrow <mattwoodrow at apple.com>
  Date:   2023-12-08 (Fri, 08 Dec 2023)

  Changed paths:
    M Source/WTF/wtf/text/TextStream.h
    M Source/WebKit/CMakeLists.txt
    M Source/WebKit/DerivedSources-input.xcfilelist
    M Source/WebKit/DerivedSources-output.xcfilelist
    M Source/WebKit/DerivedSources.make
    M Source/WebKit/GPUProcess/graphics/RemoteImageBuffer.cpp
    M Source/WebKit/GPUProcess/graphics/RemoteImageBuffer.h
    M Source/WebKit/GPUProcess/graphics/RemoteImageBuffer.messages.in
    A Source/WebKit/GPUProcess/graphics/RemoteImageBufferSet.cpp
    A Source/WebKit/GPUProcess/graphics/RemoteImageBufferSet.h
    A Source/WebKit/GPUProcess/graphics/RemoteImageBufferSet.messages.in
    M Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.cpp
    M Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.h
    M Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.messages.in
    M Source/WebKit/Platform/IPC/StreamClientConnection.h
    M Source/WebKit/Scripts/webkit/messages.py
    M Source/WebKit/Scripts/webkit/tests/MessageArgumentDescriptions.cpp
    A Source/WebKit/Shared/RemoteImageBufferSetIdentifier.h
    M Source/WebKit/Shared/RemoteLayerTree/RemoteLayerBackingStore.h
    M Source/WebKit/Shared/RemoteLayerTree/RemoteLayerBackingStore.mm
    M Source/WebKit/Shared/RemoteLayerTree/RemoteLayerBackingStoreCollection.h
    M Source/WebKit/Shared/RemoteLayerTree/RemoteLayerBackingStoreCollection.mm
    M Source/WebKit/Shared/RemoteLayerTree/RemoteLayerWithInProcessRenderingBackingStore.h
    M Source/WebKit/Shared/RemoteLayerTree/RemoteLayerWithInProcessRenderingBackingStore.mm
    M Source/WebKit/Shared/RemoteLayerTree/RemoteLayerWithRemoteRenderingBackingStore.h
    M Source/WebKit/Shared/RemoteLayerTree/RemoteLayerWithRemoteRenderingBackingStore.mm
    M Source/WebKit/Shared/RemoteLayerTree/SwapBuffersDisplayRequirement.h
    M Source/WebKit/Shared/WTFArgumentCoders.serialization.in
    M Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in
    M Source/WebKit/Sources.txt
    M Source/WebKit/SourcesCocoa.txt
    M Source/WebKit/WebKit.xcodeproj/project.pbxproj
    M Source/WebKit/WebProcess/GPU/graphics/BufferIdentifierSet.h
    M Source/WebKit/WebProcess/GPU/graphics/BufferIdentifierSet.serialization.in
    M Source/WebKit/WebProcess/GPU/graphics/PrepareBackingStoreBuffersData.cpp
    M Source/WebKit/WebProcess/GPU/graphics/PrepareBackingStoreBuffersData.h
    M Source/WebKit/WebProcess/GPU/graphics/PrepareBackingStoreBuffersData.serialization.in
    M Source/WebKit/WebProcess/GPU/graphics/RemoteDisplayListRecorderProxy.cpp
    M Source/WebKit/WebProcess/GPU/graphics/RemoteDisplayListRecorderProxy.h
    M Source/WebKit/WebProcess/GPU/graphics/RemoteImageBufferProxy.cpp
    M Source/WebKit/WebProcess/GPU/graphics/RemoteImageBufferProxy.h
    A Source/WebKit/WebProcess/GPU/graphics/RemoteImageBufferSetProxy.cpp
    A Source/WebKit/WebProcess/GPU/graphics/RemoteImageBufferSetProxy.h
    M Source/WebKit/WebProcess/GPU/graphics/RemoteRenderingBackendProxy.cpp
    M Source/WebKit/WebProcess/GPU/graphics/RemoteRenderingBackendProxy.h
    M Source/WebKit/WebProcess/GPU/graphics/RemoteRenderingBackendProxy.messages.in
    M Source/WebKit/WebProcess/Plugins/PDF/PDFPlugin.h
    M Source/WebKit/WebProcess/Plugins/PDF/UnifiedPDF/PDFDocumentLayout.h
    M Source/WebKit/WebProcess/Plugins/PDF/UnifiedPDF/UnifiedPDFPlugin.h

  Log Message:
  -----------
  Sync wait in RemoteLayerWithRemoteRenderingBackingStoreCollection::prepareBackingStoresForDisplay delays rendering.
https://bugs.webkit.org/show_bug.cgi?id=265295
<rdar://116792212>

Reviewed by Kimmo Kinnunen.

In order to start issuing rendering commands for a layer, we need to determine which ImageBuffer is not
in-use and can be drawn to. We also need to know if the existing pixel contents have been discarded due
to being marked volatile. Both of these checks can only be done in the GPU process, so the previous
behaviour was to have a synchronous call to prepareBackingStoresForDisplay at the start of rendering.

This changes makes that process asynchronous (in the common case), by moving management of the buffer
triplet (front, back, secondary back) to the GPU process, and making the choice of which buffer is used
opaque from the perspective of the web process.

For each frame, the WebProcess allocates a new image buffer identifier (via RemoteDisplayListRecorderProxy),
and the GPU process asynchronously ensures that this is mapped to the right ImageBuffer (either by picking
an available one, or allocating if needed, similar to how recycling IOSurfaces from the pool happens). It
also queues copying of the pixels from the previous frame, clipping to the new draw region, and clearing the
draw region.

The only case where this isn't possible is when the existing front buffer has been marked volatile (and
might be discarded). This should be relatively rare (and not happen during high performance animations),
and we add a sync wait to determine if the full frame needs to be redrawn or not.

This adds a new RemoteRenderingBufferSet object in the GPU process that manages the set of buffers, and
preparing backing stores, volatility setting and flushing are changed to happen in terms of the buffer set
rather than individual ImageBuffers.

We unfortuntely still have to have a main-thread sync wait at the end of the rendering update, in order to
get the mach_port_t/IOSurface handle for the front buffer used. We also need to get the set of
RenderingResourceIdentifiers used, since these are the keys for the CAIOSurface caching in the UI process.
In the common case, this message will be sent back while the WebProcess is issuing drawing commands, and
the sync wait will process the pending message immediately.
Ideally a future change will handle sending IOSurfaces (via mach_port_t) from GPUP to UI process directly,
which would remove the need for this.

The current state is a performance improvement, since we can issue rendering commands before any sync
waits, but an ideal future would remove the sync waiting from rendering entirely.

This also fixes a bug with volatility, where we can send a volatility request, then do a sync prepare for
display (which sets non volatile), and finally process the asynchronous reply from the volatility request.
In that case the web process would cache locally that the buffer is volatile and not make further requests
for it. This is fixed by tracking transaction identifiers for volatility requests to ensure that we don't
get ordering problems.

* Source/WTF/wtf/text/TextStream.h:
(WTF::operator<<):
* Source/WebKit/DerivedSources-input.xcfilelist:
* Source/WebKit/DerivedSources.make:
* Source/WebKit/GPUProcess/graphics/RemoteImageBuffer.cpp:
(WebKit::RemoteImageBuffer::create):
(WebKit::RemoteImageBuffer::RemoteImageBuffer):
* Source/WebKit/GPUProcess/graphics/RemoteImageBuffer.h:
(WebKit::RemoteImageBuffer::identifier const):
* Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.cpp:
(WebKit::RemoteRenderingBackend::didFailCreateImageBuffer):
(WebKit::RemoteRenderingBackend::didCreateImageBuffer):
(WebKit::RemoteRenderingBackend::moveToImageBuffer):
(WebKit::RemoteRenderingBackend::createImageBuffer):
(WebKit::RemoteRenderingBackend::createRemoteRenderingBufferSet):
(WebKit::RemoteRenderingBackend::releaseRemoteRenderingBufferSet):
(WebKit::RemoteRenderingBackend::updateRemoteRenderingBufferSetConfiguration):
(WebKit::RemoteRenderingBackend::createBuffersForDisplay):
(WebKit::RemoteRenderingBackend::markSurfacesVolatile):
(WebKit::handleFromBuffer): Deleted.
(WebKit::RemoteRenderingBackend::prepareBuffersForDisplay): Deleted.
(WebKit::RemoteRenderingBackend::prepareLayerBuffersForDisplay): Deleted.
* Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.h:
* Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.messages.in:
* Source/WebKit/GPUProcess/graphics/RemoteRenderingBufferSet.cpp: Added.
(WebKit::RemoteRenderingBufferSet::updateConfiguration):
(WebKit::RemoteRenderingBufferSet::ensureFrontBufferForDisplay):
(WebKit::RemoteRenderingBufferSet::prepareFrontBufferForDisplay):
(WebKit::RemoteRenderingBufferSet::makeBuffersVolatile):
* Source/WebKit/GPUProcess/graphics/RemoteRenderingBufferSet.h: Added.
(WebKit::RemoteRenderingBufferSet::isOpaque const):
* Source/WebKit/Platform/IPC/StreamClientConnection.h:
(IPC::StreamClientConnection::waitForAsyncReplyAndDispatchImmediately):
* Source/WebKit/Scripts/webkit/messages.py:
(serialized_identifiers):
(class_template_headers):
(headers_for_type):
* Source/WebKit/Shared/RemoteBufferSetIdentifier.h: Copied from Source/WebKit/WebProcess/GPU/graphics/BufferIdentifierSet.h.
* Source/WebKit/Shared/RemoteLayerTree/BufferAndBackendInfo.h: Removed.
* Source/WebKit/Shared/RemoteLayerTree/BufferAndBackendInfo.serialization.in: Removed.
* Source/WebKit/Shared/RemoteLayerTree/RemoteLayerBackingStore.h:
(WebKit::RemoteLayerBackingStore::dirtyRegion):
(WebKit::RemoteLayerBackingStore::setBufferCacheIdentifiers):
* Source/WebKit/Shared/RemoteLayerTree/RemoteLayerBackingStore.mm:
(WebKit::RemoteLayerBackingStore::ensureBackingStore):
(WebKit::RemoteLayerBackingStore::encode const):
(WebKit::RemoteLayerBackingStoreProperties::decode):
(WebKit::RemoteLayerBackingStoreProperties::dump const):
(WebKit::RemoteLayerBackingStore::setFrontBuffer):
(WebKit::RemoteLayerBackingStore::supportsPartialRepaint const):
(WebKit::RemoteLayerBackingStore::drawingRequiresClearedPixels const):
(WebKit::RemoteLayerBackingStore::setDelegatedContents):
(WebKit::RemoteLayerBackingStore::paintContents):
(WebKit::RemoteLayerBackingStore::drawInContext):
(WebKit::RemoteLayerBackingStoreProperties::applyBackingStoreToLayer):
(WebKit::RemoteLayerBackingStoreProperties::updateCachedBuffers):
(WebKit::RemoteLayerBackingStore::applySwappedBuffers): Deleted.
* Source/WebKit/Shared/RemoteLayerTree/RemoteLayerBackingStoreCollection.h:
(WebKit::RemoteLayerBackingStoreCollection::backingStoreUpdated):
(WebKit::RemoteLayerBackingStoreCollection::backingStoreWillBeEncoded):
* Source/WebKit/Shared/RemoteLayerTree/RemoteLayerTreePropertyApplier.mm:
(WebKit::RemoteLayerTreePropertyApplier::applyPropertiesToLayer):
* Source/WebKit/Shared/RemoteLayerTree/RemoteLayerWithRemoteRenderingBackingStoreCollection.h:
* Source/WebKit/Shared/RemoteLayerTree/RemoteLayerWithRemoteRenderingBackingStoreCollection.mm:
(WebKit::RemoteLayerWithRemoteRenderingBackingStoreCollection::paintReachableBackingStoreContents):
(WebKit::RemoteLayerWithRemoteRenderingBackingStoreCollection::prepareBackingStoresForDisplay):
(WebKit::RemoteLayerWithRemoteRenderingBackingStoreCollection::allocateBufferForBackingStore):
(WebKit::RemoteLayerWithRemoteRenderingBackingStoreCollection::collectBackingStoreBufferIdentifiersToMarkVolatile):
(WebKit::RemoteLayerWithRemoteRenderingBackingStoreCollection::collectAllBufferIdentifiersToMarkVolatile):
(WebKit::RemoteLayerWithRemoteRenderingBackingStoreCollection::markBackingStoreVolatileAfterReachabilityChange):
(WebKit::RemoteLayerWithRemoteRenderingBackingStoreCollection::tryMarkAllBackingStoreVolatile):
(WebKit::RemoteLayerWithRemoteRenderingBackingStoreCollection::markAllBackingStoreVolatileFromTimer):
(WebKit::RemoteLayerWithRemoteRenderingBackingStoreCollection::sendMarkBuffersVolatile):
(WebKit::RemoteLayerWithRemoteRenderingBackingStoreCollection::gpuProcessConnectionWasDestroyed):
(WebKit::RemoteLayerWithRemoteRenderingBackingStoreCollection::backingStoreWasCreated):
(WebKit::RemoteLayerWithRemoteRenderingBackingStoreCollection::backingStoreWillBeDestroyed):
(WebKit::RemoteLayerWithRemoteRenderingBackingStoreCollection::backingStoreUpdated):
(WebKit::RemoteLayerWithRemoteRenderingBackingStoreCollection::backingStoreWillBeEncoded):
* Source/WebKit/Shared/WTFArgumentCoders.serialization.in:
* Source/WebKit/Sources.txt:
* Source/WebKit/UIProcess/RemoteLayerTree/RemoteLayerTreeDrawingAreaProxy.h:
* Source/WebKit/UIProcess/RemoteLayerTree/RemoteLayerTreeDrawingAreaProxy.messages.in:
* Source/WebKit/UIProcess/RemoteLayerTree/RemoteLayerTreeDrawingAreaProxy.mm:
(WebKit::RemoteLayerTreeDrawingAreaProxy::asyncSetLayerContents):
* Source/WebKit/UIProcess/RemoteLayerTree/RemoteLayerTreeHost.h:
* Source/WebKit/UIProcess/RemoteLayerTree/RemoteLayerTreeHost.mm:
(WebKit::RemoteLayerTreeHost::asyncSetLayerContents):
* Source/WebKit/UIProcess/RemoteLayerTree/RemoteLayerTreeNode.h:
(WebKit::RemoteLayerTreeNode::asyncContentsIdentifier const):
(WebKit::RemoteLayerTreeNode::setAsyncContentsIdentifier):
* Source/WebKit/WebKit.xcodeproj/project.pbxproj:
* Source/WebKit/WebProcess/GPU/graphics/BufferIdentifierSet.h:
(WebKit::operator<<):
* Source/WebKit/WebProcess/GPU/graphics/BufferIdentifierSet.serialization.in:
* Source/WebKit/WebProcess/GPU/graphics/PrepareBackingStoreBuffersData.cpp:
(WebKit::operator<<):
* Source/WebKit/WebProcess/GPU/graphics/PrepareBackingStoreBuffersData.h:
* Source/WebKit/WebProcess/GPU/graphics/PrepareBackingStoreBuffersData.serialization.in:
* Source/WebKit/WebProcess/GPU/graphics/RemoteImageBufferProxy.cpp:
(WebKit::RemoteImageBufferProxy::RemoteImageBufferProxy):
(WebKit::m_remoteBufferManager):
(WebKit::RemoteImageBufferProxy::remoteBufferWasDestroyed):
(WebKit::RemoteSerializedImageBufferProxy::sinkIntoImageBuffer):
* Source/WebKit/WebProcess/GPU/graphics/RemoteImageBufferProxy.h:
(WebKit::RemoteImageBufferProxy::create):
* Source/WebKit/WebProcess/GPU/graphics/RemoteRenderingBackendProxy.cpp:
(WebKit::RemoteRenderingBackendProxy::didClose):
(WebKit::RemoteRenderingBackendProxy::createImageBuffer):
(WebKit::RemoteRenderingBackendProxy::createRemoteRenderingBufferSet):
(WebKit::RemoteRenderingBackendProxy::releaseRemoteRenderingBufferSet):
(WebKit::RemoteRenderingBackendProxy::updateRemoteRenderingBufferSetConfiguration):
(WebKit::RemoteRenderingBackendProxy::prepareBuffersForDisplay):
(WebKit::RemoteRenderingBackendProxy::ensurePrepareCompleted):
(WebKit::RemoteRenderingBackendProxy::markSurfacesVolatile):
(WebKit::RemoteRenderingBackendProxy::didMarkLayersAsVolatile):
* Source/WebKit/WebProcess/GPU/graphics/RemoteRenderingBackendProxy.h:
* Source/WebKit/WebProcess/GPU/graphics/RemoteRenderingBackendProxy.messages.in:
* Source/WebKit/WebProcess/GPU/graphics/RemoteRenderingBufferSetProxy.cpp: Copied from Source/WebKit/WebProcess/GPU/graphics/BufferIdentifierSet.h.
(WebKit::RemoteRenderingBufferSetProxy::RemoteRenderingBufferSetProxy):
(WebKit::RemoteRenderingBufferSetProxy::~RemoteRenderingBufferSetProxy):
* Source/WebKit/WebProcess/GPU/graphics/RemoteRenderingBufferSetProxy.h: Added.
(WebKit::RemoteRenderingBufferSetProxy::remoteBufferSetIdentifier const):
(WebKit::RemoteRenderingBufferSetProxy::requestedVolatility):
(WebKit::RemoteRenderingBufferSetProxy::confirmedVolatility):
(WebKit::RemoteRenderingBufferSetProxy::addRequestedVolatility):
(WebKit::RemoteRenderingBufferSetProxy::setConfirmedVolatility):
(WebKit::RemoteRenderingBufferSetProxy::clearVolatilityUntilAfter):
(WebKit::operator<<):
* Source/WebKit/WebProcess/GPU/graphics/RemoteResourceCacheProxy.cpp:
(WebKit::RemoteResourceCacheProxy::remoteResourceCacheWasDestroyed):
* Source/WebKit/WebProcess/Plugins/PDF/PDFPlugin.h:
* Source/WebKit/WebProcess/Plugins/PDF/UnifiedPDF/PDFDocumentLayout.h:
* Source/WebKit/WebProcess/Plugins/PDF/UnifiedPDF/UnifiedPDFPlugin.h:
* Source/WebKit/WebProcess/WebPage/RemoteLayerTree/GraphicsLayerCARemote.mm:

Canonical link: https://commits.webkit.org/271747@main

More information about the webkit-changes mailing list