[webkit-changes] [WebKit/WebKit] 10ea3b: Extend AudioSampleBufferCompressor capabilities to...

[Jean-Yves Avenard]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 10ea3bdfd9009b070aa95cb6e46e7948f0b1b7f6
      https://github.com/WebKit/WebKit/commit/10ea3bdfd9009b070aa95cb6e46e7948f0b1b7f6
  Author: Jean-Yves Avenard <jya at apple.com>
  Date:   2024-12-04 (Wed, 04 Dec 2024)

  Changed paths:
    M Source/WebCore/platform/audio/cocoa/AudioFileReaderCocoa.cpp
    M Source/WebCore/platform/audio/cocoa/AudioFileReaderCocoa.h
    M Source/WebCore/platform/graphics/cocoa/CMUtilities.h
    M Source/WebCore/platform/graphics/cocoa/CMUtilities.mm
    M Source/WebCore/platform/mediarecorder/MediaRecorderPrivateEncoder.cpp
    M Source/WebCore/platform/mediarecorder/cocoa/AudioSampleBufferCompressor.h
    M Source/WebCore/platform/mediarecorder/cocoa/AudioSampleBufferCompressor.mm

  Log Message:
  -----------
  Extend AudioSampleBufferCompressor capabilities to allow for any CMSampleBuffer conversion
https://bugs.webkit.org/show_bug.cgi?id=283905
rdar://140782536

Reviewed by Youenn Fablet.

We extend the class to be able to decode various format into PCM as required by the WebCodec's AudioDecoderCocoa class
(WebCodec's AudioEncoderCocoa).
We also rewrite the AudioConverterComplexInputDataProc used by processSample using
the available CoreMedia methods and how we create a CMSampleBuffer. This new version is significantly
smaller and less complex while being able to handle more format (in particular, in can now handle any planar formats and not a maximum of two).

The addSampleBuffer method now returns a GenericPromise which is resolved
once the sampleBuffer has been fulled decoded and all data has been returned
to the callback.
Decoding or Encoding errors are now reported back to the consumer/

We add a new mode where the converter will now longer generate the timestamps but
instead set the timestamp of the input to be the same as the one on the input
(this is an AudioDecoder requirement)

This change will be fully tested in a follow-up commit where we introduce AudioDecoderCocoa.
For now, there's no impact on its only consumer: MediaRecorderPrivateEncoder
which is covered by existing tests.

Fly-By: change use of `auto` for explicit `int` when reporting error as it causes compilation error when using OSStatus.

* Source/WebCore/platform/audio/cocoa/AudioFileReaderCocoa.cpp:
(WebCore::AudioFileReader::getPacketDescriptions const): Deleted.
* Source/WebCore/platform/audio/cocoa/AudioFileReaderCocoa.h:
* Source/WebCore/platform/graphics/cocoa/CMUtilities.h: Extract getPacketDescriptions method from AudioFileReaderCocoa.
* Source/WebCore/platform/graphics/cocoa/CMUtilities.mm:
(WebCore::getPacketDescriptions):
* Source/WebCore/platform/mediarecorder/cocoa/AudioSampleBufferCompressor.h:
(WebCore::AudioSampleBufferCompressor::isPCM const):
* Source/WebCore/platform/mediarecorder/cocoa/AudioSampleBufferCompressor.mm:
(WebCore::AudioSampleBufferCompressor::flushInternal):
(WebCore::AudioSampleBufferCompressor::setGenerateTimestamp):
(WebCore::AudioSampleBufferCompressor::initAudioConverterForSourceFormatDescription):
(WebCore::AudioSampleBufferCompressor::gradualDecoderRefreshCount):
(WebCore::AudioSampleBufferCompressor::sampleBuffer):
(WebCore::AudioSampleBufferCompressor::provideSourceDataNumOutputPackets):
(WebCore::AudioSampleBufferCompressor::setTimeFromSample):
(WebCore::AudioSampleBufferCompressor::processSampleBuffers):
(WebCore::AudioSampleBufferCompressor::addSampleBuffer):

Canonical link: https://commits.webkit.org/287362@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications