[webkit-changes] [WebKit/WebKit] 678b80: CARingBuffer uses CAAudioStreamDescription in a re...

Kimmo Kinnunen noreply at github.com
Mon Nov 7 02:29:49 PST 2022


  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 678b80c4edb8be97cb21b7f1141c9b154c50b591
      https://github.com/WebKit/WebKit/commit/678b80c4edb8be97cb21b7f1141c9b154c50b591
  Author: Kimmo Kinnunen <kkinnunen at apple.com>
  Date:   2022-11-07 (Mon, 07 Nov 2022)

  Changed paths:
    M Source/WebCore/platform/audio/cocoa/AudioSampleBufferList.h
    M Source/WebCore/platform/audio/cocoa/AudioSampleDataSource.h
    M Source/WebCore/platform/audio/cocoa/AudioSampleDataSource.mm
    M Source/WebCore/platform/audio/cocoa/CARingBuffer.cpp
    M Source/WebCore/platform/audio/cocoa/CARingBuffer.h
    M Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm
    M Source/WebKit/GPUProcess/media/RemoteAudioDestinationManager.cpp
    M Source/WebKit/GPUProcess/media/RemoteAudioSourceProviderProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteAudioSourceProviderProxy.h
    M Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererInternalUnitManager.cpp
    M Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.cpp
    M Source/WebKit/Platform/SharedMemory.h
    M Source/WebKit/Platform/cocoa/SharedMemoryCocoa.cpp
    M Source/WebKit/Platform/unix/SharedMemoryUnix.cpp
    M Source/WebKit/Platform/win/SharedMemoryWin.cpp
    M Source/WebKit/Shared/Cocoa/SharedCARingBuffer.cpp
    M Source/WebKit/Shared/Cocoa/SharedCARingBuffer.h
    M Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp
    M Source/WebKit/UIProcess/SpeechRecognitionRemoteRealtimeMediaSource.cpp
    M Source/WebKit/WebProcess/GPU/media/RemoteAudioDestinationProxy.cpp
    M Source/WebKit/WebProcess/GPU/media/RemoteAudioDestinationProxy.h
    M Source/WebKit/WebProcess/GPU/media/RemoteAudioSourceProviderManager.cpp
    M Source/WebKit/WebProcess/GPU/webrtc/AudioMediaStreamTrackRendererInternalUnitManager.cpp
    M Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp
    M Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.h
    M Source/WebKit/WebProcess/Speech/SpeechRecognitionRealtimeMediaSourceManager.cpp
    M Source/WebKit/WebProcess/cocoa/RemoteCaptureSampleManager.cpp
    M Tools/TestWebKitAPI/Tests/WebCore/CARingBuffer.cpp

  Log Message:
  -----------
  CARingBuffer uses CAAudioStreamDescription in a redundant way
https://bugs.webkit.org/show_bug.cgi?id=247430
rdar://problem/101914553

Reviewed by Eric Carlson.

CARingBuffer structure is defined by channelCount, bytesPerFrame, frameCount. However,
it also stores CAAudioStreamDescription to define the contents of each frame
in case client wants to mix (contents can be floats, ints of different size).
CARingBuffer might also be used to transfer undefined frames (possibly non-PCM data), via the
copy operation.

Using CAAudioStreamDescription is problematic, as it is redundant information and it smuggles
the non-PCM data definition via the platform-specific descriptor which is not consistent.
In order to stop using CAAudioStreamDescription in the future patches, remove it from
CARingBuffer. The fetch() caller already knows what kind of data is expected, so this information
is redundant. This fixes the problem that the caller expectation and what is inside the
CARingBuffer could diverge.

Leave the use of CAAudioStreamDescription in the constructors, but remove it from the
internals of the classes.

This patch would affect the base-class constructors and functions. Take the opportunity
to change the CARingBuffer allocation logic: previously, one would construct an empty
instance and then at some point call allocate(). In case the allocate would fail, the
instance would stay "empty" with quite undefined behavior. Instead, make all CARingBuffer
classes be normal memory buffer classes: creation will allocate the buffer, and this
might fail. Deletion will deallocate.

Previously ProducerSharedCARingBuffer allocation sequence was a bit convoluted:
- WP proxy class would construct empty instance with a resize callback.
- When allocate and deallocate would succeed, the resize callback would be called.
- The callers would construct with a resize callback that would then
  send the "audio storage changed" message to the other process, typically GPUP.
The sequence would be linear, resize callback invocation would always follow
an allocate. However, it was written in non-linear fashion.

Instead just have a linear logic:
- WP proxy class constructs the shared buffer, and gets the buffer and
  client handle as a result.
- WP proxy class sends the client handle to the other process.

In order to return ConsumerSharedCARingBuffer::Handle (alias for
SharedMemory::Handle) as part of a struct, make it copyable.
Recent changes to SharedMemory::Handle enable this and were made
to for this purpose.

* Source/WebCore/platform/audio/cocoa/AudioSampleBufferList.h:
* Source/WebCore/platform/audio/cocoa/AudioSampleDataSource.h:
* Source/WebCore/platform/audio/cocoa/AudioSampleDataSource.mm:
(WebCore::AudioSampleDataSource::setOutputFormat):
(WebCore::AudioSampleDataSource::pushSamplesInternal):
(WebCore::AudioSampleDataSource::pullSamples):
(WebCore::AudioSampleDataSource::pullSamplesInternal):
(WebCore::AudioSampleDataSource::pullAvailableSampleChunk):
(WebCore::AudioSampleDataSource::pullAvailableSamplesAsChunks):
* Source/WebCore/platform/audio/cocoa/CARingBuffer.cpp:
(WebCore::CARingBuffer::CARingBuffer):
(WebCore::CARingBuffer::computeCapacityBytes):
(WebCore::CARingBuffer::computeSizeForBuffers):
(WebCore::CARingBuffer::initialize):
(WebCore::FetchABL):
(WebCore::CARingBuffer::fetchInternal):
(WebCore::InProcessCARingBuffer::allocate):
(WebCore::InProcessCARingBuffer::InProcessCARingBuffer):
(WebCore::CARingBuffer::initializeAfterAllocation): Deleted.
(WebCore::CARingBuffer::allocate): Deleted.
(WebCore::CARingBuffer::deallocate): Deleted.
(WebCore::InProcessCARingBuffer::allocateBuffers): Deleted.
* Source/WebCore/platform/audio/cocoa/CARingBuffer.h:
(WebCore::CARingBuffer::fetchModeForMixing):
* Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.h:
* Source/WebCore/platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm:
(WebCore::AudioSourceProviderAVFObjC::AudioSourceProviderAVFObjC):
(WebCore::AudioSourceProviderAVFObjC::prepare):
(WebCore::AudioSourceProviderAVFObjC::process):
(WebCore::AudioSourceProviderAVFObjC::setConfigureAudioStorageCallback):
(WebCore::AudioSourceProviderAVFObjC::setRingBufferCreationCallback): Deleted.
* Source/WebKit/GPUProcess/media/RemoteAudioDestinationManager.cpp:
* Source/WebKit/GPUProcess/media/RemoteAudioSourceProviderProxy.cpp:
(WebKit::RemoteAudioSourceProviderProxy::create):
(WebKit::RemoteAudioSourceProviderProxy::configureAudioStorage):
(WebKit::RemoteAudioSourceProviderProxy::createRingBuffer): Deleted.
(WebKit::RemoteAudioSourceProviderProxy::storageChanged): Deleted.
* Source/WebKit/GPUProcess/media/RemoteAudioSourceProviderProxy.h:
* Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererInternalUnitManager.cpp:
(WebKit::RemoteAudioMediaStreamTrackRendererInternalUnitManager::Unit::start):
* Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.cpp:
(WebKit::RemoteMediaRecorder::audioSamplesStorageChanged):
* Source/WebKit/Platform/SharedMemory.h:
* Source/WebKit/Platform/cocoa/SharedMemoryCocoa.cpp:
* Source/WebKit/Platform/unix/SharedMemoryUnix.cpp:
(WebKit::SharedMemory::Handle::Handle): Deleted.
(WebKit::SharedMemory::Handle::~Handle): Deleted.
* Source/WebKit/Platform/win/SharedMemoryWin.cpp:
* Source/WebKit/Shared/Cocoa/SharedCARingBuffer.cpp:
(WebKit::SharedCARingBufferBase::SharedCARingBufferBase):
(WebKit::SharedCARingBufferBase::data):
(WebKit::SharedCARingBufferBase::sharedFrameBounds const):
(WebKit::SharedCARingBufferBase::updateFrameBounds):
(WebKit::SharedCARingBufferBase::size const):
(WebKit::ConsumerSharedCARingBuffer::map):
(WebKit::ProducerSharedCARingBuffer::allocate):
(WebKit::ProducerSharedCARingBuffer::setCurrentFrameBounds):
(WebKit::ConsumerSharedCARingBuffer::ConsumerSharedCARingBuffer): Deleted.
(WebKit::ConsumerSharedCARingBuffer::allocateBuffers): Deleted.
(WebKit::ProducerSharedCARingBuffer::setStorage): Deleted.
(WebKit::ProducerSharedCARingBuffer::allocateBuffers): Deleted.
(WebKit::ProducerSharedCARingBuffer::deallocateBuffers): Deleted.
* Source/WebKit/Shared/Cocoa/SharedCARingBuffer.h:
* Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp:
(WebKit::UserMediaCaptureManagerProxy::SourceProxy::~SourceProxy):
(WebKit::UserMediaCaptureManagerProxy::SourceProxy::storageChanged): Deleted.
* Source/WebKit/UIProcess/SpeechRecognitionRemoteRealtimeMediaSource.cpp:
(WebKit::SpeechRecognitionRemoteRealtimeMediaSource::setStorage):
* Source/WebKit/WebProcess/GPU/media/RemoteAudioDestinationProxy.cpp:
(WebKit::RemoteAudioDestinationProxy::RemoteAudioDestinationProxy):
(WebKit::RemoteAudioDestinationProxy::connection):
(WebKit::RemoteAudioDestinationProxy::storageChanged): Deleted.
* Source/WebKit/WebProcess/GPU/media/RemoteAudioDestinationProxy.h:
* Source/WebKit/WebProcess/GPU/media/RemoteAudioSourceProviderManager.cpp:
(WebKit::RemoteAudioSourceProviderManager::RemoteAudio::setStorage):
* Source/WebKit/WebProcess/GPU/webrtc/AudioMediaStreamTrackRendererInternalUnitManager.cpp:
(WebKit::AudioMediaStreamTrackRendererInternalUnitManager::Proxy::start):
(WebKit::AudioMediaStreamTrackRendererInternalUnitManager::Proxy::storageChanged): Deleted.
* Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp:
(WebKit::MediaRecorderPrivate::startRecording):
(WebKit::MediaRecorderPrivate::audioSamplesAvailable):
(WebKit::MediaRecorderPrivate::storageChanged): Deleted.
* Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.h:
* Source/WebKit/WebProcess/Speech/SpeechRecognitionRealtimeMediaSourceManager.cpp:
(WebKit::SpeechRecognitionRealtimeMediaSourceManager::Source::Source):
(WebKit::SpeechRecognitionRealtimeMediaSourceManager::Source::~Source):
(WebKit::SpeechRecognitionRealtimeMediaSourceManager::Source::storageChanged): Deleted.
* Source/WebKit/WebProcess/cocoa/RemoteCaptureSampleManager.cpp:
(WebKit::RemoteCaptureSampleManager::RemoteAudio::setStorage):
* Tools/TestWebKitAPI/Tests/WebCore/CARingBuffer.cpp:
(TestWebKitAPI::CARingBufferTest::SetUp):
(TestWebKitAPI::CARingBufferTest::setup):
(TestWebKitAPI::MixingTest::run):

Canonical link: https://commits.webkit.org/256392@main




More information about the webkit-changes mailing list