[webkit-changes] [WebKit/WebKit] cb7682: MediaRecorder shouldn't perform memory allocation ...

[Jean-Yves Avenard]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: cb76829ab268a109558cca5efdce6ca18913e643
      https://github.com/WebKit/WebKit/commit/cb76829ab268a109558cca5efdce6ca18913e643
  Author: Jean-Yves Avenard <jya at apple.com>
  Date:   2024-11-04 (Mon, 04 Nov 2024)

  Changed paths:
    M Source/WebCore/platform/audio/cocoa/WebAudioBufferList.cpp
    M Source/WebCore/platform/audio/cocoa/WebAudioBufferList.h
    M Source/WebCore/platform/mediarecorder/MediaRecorderPrivateEncoder.cpp
    M Source/WebCore/platform/mediarecorder/MediaRecorderPrivateEncoder.h

  Log Message:
  -----------
  MediaRecorder shouldn't perform memory allocation on the thread it is receiving audio samples
https://bugs.webkit.org/show_bug.cgi?id=282374
rdar://problem/138978311

Reviewed by Youenn Fablet.

We used to create a CMSampleBuffer in the audio thread before sending it to the AudioSampleBufferCompressor
for encoding.

We now instead allocate an InProcessCARingBuffer when we receive the first audio sample.
>From that point, we push the new audio in the ringbuffer and send a message to the encoder's WorkQueue
to retrieve the data where the CMSampleBuffer will be created.
This reduces the number of memory allocation to once only.
The CARingBuffer is made to allow for 2s of audio. This is the same size used by the GPU process
to capture the audio and send it to the content process in its own ringbuffer.

No change in observable behaviour. Covered by existing tests.

* Source/WebCore/platform/audio/cocoa/WebAudioBufferList.cpp:
(WebCore::WebAudioBufferList::setSampleCount):
(WebCore::WebAudioBufferList::createWebAudioBufferListWithBlockBuffer):
(WebCore::WebAudioBufferList::setSampleCountWithBlockBuffer):
(WebCore::WebAudioBufferList::initializeList):
* Source/WebCore/platform/audio/cocoa/WebAudioBufferList.h: We add a new method to create an AudioBufferList backed by a CMBlockBoffer.
This allows to directly copy the data from the CARingBuffer to the CMSampleBuffer without an intermediary.
* Source/WebCore/platform/mediarecorder/MediaRecorderPrivateEncoder.cpp:
(WebCore::MediaRecorderPrivateEncoder::appendAudioSampleBuffer):
(WebCore::MediaRecorderPrivateEncoder::audioSamplesAvailableStarted):
(WebCore::MediaRecorderPrivateEncoder::audioSamplesAvailable):
(WebCore::MediaRecorderPrivateEncoder::appendVideoFrame): Fly-By: check if recorder is stopped before enqueuing. So that even in-flight video frames
at the time the recorder was stopped will be processed and converted.
(WebCore::MediaRecorderPrivateEncoder::enqueueCompressedAudioSampleBuffers):
(WebCore::MediaRecorderPrivateEncoder::maybeStartWriter):
(WebCore::MediaRecorderPrivateEncoder::stopRecording):
* Source/WebCore/platform/mediarecorder/MediaRecorderPrivateEncoder.h:
(WebCore::MediaRecorderPrivateEncoder::hasAudio const): Deleted.
(WebCore::MediaRecorderPrivateEncoder::hasVideo const): Deleted.
(WebCore::MediaRecorderPrivateEncoder::WTF_GUARDED_BY_CAPABILITY): Deleted.

Canonical link: https://commits.webkit.org/286083@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications