[webkit-changes] [WebKit/WebKit] d27040: [Cocoa] Add webm support to MediaRecorder

Fri Oct 11 23:13:36 PDT 2024

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: d27040f0d729453dbf87696eddd077ec1f6d25d3
      https://github.com/WebKit/WebKit/commit/d27040f0d729453dbf87696eddd077ec1f6d25d3
  Author: Jean-Yves Avenard <jya at apple.com>
  Date:   2024-10-11 (Fri, 11 Oct 2024)

  Changed paths:
    M Configurations/WebKitProjectPaths.xcconfig
    M LayoutTests/http/wpt/mediarecorder/MediaRecorder-AV-audio-video-dataavailable-gpuprocess-expected.txt
    M LayoutTests/http/wpt/mediarecorder/MediaRecorder-AV-audio-video-dataavailable-gpuprocess.html
    M LayoutTests/http/wpt/mediarecorder/MediaRecorder-audio-bitrate-expected.txt
    M LayoutTests/http/wpt/mediarecorder/MediaRecorder-audio-bitrate.html
    M LayoutTests/http/wpt/mediarecorder/MediaRecorder-requestData-expected.txt
    M LayoutTests/http/wpt/mediarecorder/MediaRecorder-requestData.html
    M LayoutTests/http/wpt/mediarecorder/mimeType-expected.txt
    M LayoutTests/http/wpt/mediarecorder/mimeType.html
    M LayoutTests/http/wpt/mediarecorder/pause-recording-expected.txt
    M LayoutTests/http/wpt/mediarecorder/pause-recording.html
    M LayoutTests/imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-mimetype-expected.txt
    M LayoutTests/imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https-expected.txt
    M LayoutTests/platform/glib/TestExpectations
    M Source/ThirdParty/libwebrtc/CMakeLists.txt
    M Source/ThirdParty/libwebrtc/Configurations/BaseTarget-libwebm.xcconfig
    M Source/ThirdParty/libwebrtc/Configurations/libwebm.xcconfig
    M Source/ThirdParty/libwebrtc/Configurations/libwebrtc.exp
    M Source/ThirdParty/libwebrtc/Source/third_party/libwebm/CMakeLists.txt
    M Source/ThirdParty/libwebrtc/libwebrtc.xcodeproj/project.pbxproj
    M Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml
    M Source/WTF/wtf/PlatformEnable.h
    M Source/WTF/wtf/PlatformEnableCocoa.h
    M Source/WebCore/Modules/mediarecorder/MediaRecorder.cpp
    M Source/WebCore/Modules/mediarecorder/MediaRecorderProvider.cpp
    M Source/WebCore/Modules/mediarecorder/MediaRecorderProvider.h
    M Source/WebCore/PlatformMac.cmake
    M Source/WebCore/SourcesCocoa.txt
    M Source/WebCore/WebCore.xcodeproj/project.pbxproj
    M Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.h
    M Source/WebCore/platform/graphics/cocoa/VP9UtilitiesCocoa.mm
    M Source/WebCore/platform/graphics/cocoa/WebMAudioUtilitiesCocoa.h
    M Source/WebCore/platform/graphics/cocoa/WebMAudioUtilitiesCocoa.mm
    M Source/WebCore/platform/mediarecorder/MediaRecorderPrivateAVFImpl.cpp
    A Source/WebCore/platform/mediarecorder/MediaRecorderPrivateWriter.cpp
    A Source/WebCore/platform/mediarecorder/MediaRecorderPrivateWriter.h
    M Source/WebCore/platform/mediarecorder/cocoa/AudioSampleBufferCompressor.h
    M Source/WebCore/platform/mediarecorder/cocoa/AudioSampleBufferCompressor.mm
    M Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h
    M Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm
    A Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterWebM.h
    A Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterWebM.mm
    M Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.cpp
    M Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.h
    M Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.messages.in
    M Source/WebKit/UIProcess/Cocoa/PlaybackSessionManagerProxy.mm
    M Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp

  Log Message:
  -----------
  [Cocoa] Add webm support to MediaRecorder
https://bugs.webkit.org/show_bug.cgi?id=281106
rdar://137560454

Reviewed by Youenn Fablet.

We add a MediaRecorderPrivateWriterWebM that uses the mkvmuxer from libwebm and the WebCodecs video encoders.
The WebM MediaRecorder is behind a preferences, currently disabled by default.

The MediaRecorderPrivateWriterWebM is currently used in the GPU process, in place of the MediaRecorderPrivateWriterCocoa
for ease of integration.
Performing the encoding and muxing will be moved to the Web Process in a follow-up change.

Amending existing tests to exercise the code.

* Configurations/WebKitProjectPaths.xcconfig:
* LayoutTests/http/wpt/mediarecorder/MediaRecorder-AV-audio-video-dataavailable-gpuprocess-expected.txt:
* LayoutTests/http/wpt/mediarecorder/MediaRecorder-AV-audio-video-dataavailable-gpuprocess.html:
* LayoutTests/http/wpt/mediarecorder/MediaRecorder-audio-bitrate-expected.txt:
* LayoutTests/http/wpt/mediarecorder/MediaRecorder-audio-bitrate.html:
* LayoutTests/http/wpt/mediarecorder/MediaRecorder-requestData-expected.txt:
* LayoutTests/http/wpt/mediarecorder/MediaRecorder-requestData.html:
* LayoutTests/http/wpt/mediarecorder/mimeType-expected.txt:
* LayoutTests/http/wpt/mediarecorder/mimeType.html:
* LayoutTests/http/wpt/mediarecorder/pause-recording-expected.txt:
* LayoutTests/http/wpt/mediarecorder/pause-recording.html:
* LayoutTests/imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-mimetype-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https-expected.txt:
* LayoutTests/platform/glib/TestExpectations: GStreamer MediaRecorder always create a MP4 one.
* Source/ThirdParty/libwebrtc/CMakeLists.txt:
* Source/ThirdParty/libwebrtc/Configurations/BaseTarget-libwebm.xcconfig:
* Source/ThirdParty/libwebrtc/Configurations/libwebm.xcconfig:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.exp:
* Source/ThirdParty/libwebrtc/Source/third_party/libwebm/CMakeLists.txt:
* Source/ThirdParty/libwebrtc/libwebrtc.xcodeproj/project.pbxproj:
* Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml:
* Source/WTF/wtf/PlatformEnable.h:
* Source/WTF/wtf/PlatformEnableCocoa.h:
* Source/WebCore/Modules/mediarecorder/MediaRecorder.cpp:
(WebCore::MediaRecorder::isTypeSupported):
* Source/WebCore/Modules/mediarecorder/MediaRecorderProvider.cpp:
(WebCore::MediaRecorderProvider::isSupported):
(WebCore::MediaRecorderProvider::isWebMAndSupported):
* Source/WebCore/Modules/mediarecorder/MediaRecorderProvider.h:
* Source/WebCore/PlatformMac.cmake:
* Source/WebCore/SourcesCocoa.txt:
* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.h:
* Source/WebCore/platform/graphics/cocoa/VP9UtilitiesCocoa.mm:
* Source/WebCore/platform/graphics/cocoa/WebMAudioUtilitiesCocoa.h:
* Source/WebCore/platform/graphics/cocoa/WebMAudioUtilitiesCocoa.mm:
(WebCore::span8):
(WebCore::createOpusPrivateData):
* Source/WebCore/platform/mediarecorder/MediaRecorderPrivateAVFImpl.cpp:
(WebCore::MediaRecorderPrivateAVFImpl::create):
* Source/WebCore/platform/mediarecorder/MediaRecorderPrivateWriter.cpp: Copied from Source/WebCore/Modules/mediarecorder/MediaRecorderProvider.h.
(WebCore::MediaRecorderPrivateWriter::create):
* Source/WebCore/platform/mediarecorder/MediaRecorderPrivateWriter.h: Added.
* Source/WebCore/platform/mediarecorder/cocoa/AudioSampleBufferCompressor.h:
* Source/WebCore/platform/mediarecorder/cocoa/AudioSampleBufferCompressor.mm:
(WebCore::AudioSampleBufferCompressor::create):
(WebCore::AudioSampleBufferCompressor::AudioSampleBufferCompressor):
(WebCore::AudioSampleBufferCompressor::initAudioConverterForSourceFormatDescription):
(WebCore::AudioSampleBufferCompressor::getOutputSampleBuffer const):
(WebCore::AudioSampleBufferCompressor::isEmpty const):
(WebCore::AudioSampleBufferCompressor::getOutputSampleBuffer): Deleted.
* Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h:
* Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm:
(WebCore::MediaRecorderPrivateWriter::create):
(WebCore::MediaRecorderPrivateWriterAVFObjC::create):
(WebCore::MediaRecorderPrivateWriterAVFObjC::compressedVideoOutputBufferCallback):
(WebCore::MediaRecorderPrivateWriterAVFObjC::compressedAudioOutputBufferCallback):
(WebCore::MediaRecorderPrivateWriterAVFObjC::MediaRecorderPrivateWriterAVFObjC):
(WebCore::MediaRecorderPrivateWriterAVFObjC::~MediaRecorderPrivateWriterAVFObjC):
(WebCore::MediaRecorderPrivateWriterAVFObjC::close):
(WebCore::MediaRecorderPrivateWriterAVFObjC::initialize):
(WebCore::MediaRecorderPrivateWriterAVFObjC::processNewCompressedVideoSampleBuffers):
(WebCore::MediaRecorderPrivateWriterAVFObjC::processNewCompressedAudioSampleBuffers):
(WebCore::MediaRecorderPrivateWriterAVFObjC::startAssetWriter):
(WebCore::MediaRecorderPrivateWriterAVFObjC::appendCompressedAudioSampleBufferIfPossible):
(WebCore::MediaRecorderPrivateWriterAVFObjC::appendCompressedVideoSampleBufferIfPossible):
(WebCore::MediaRecorderPrivateWriterAVFObjC::appendCompressedVideoSampleBuffer):
(WebCore::MediaRecorderPrivateWriterAVFObjC::appendCompressedSampleBuffers):
(WebCore::MediaRecorderPrivateWriterAVFObjC::flushCompressedSampleBuffers):
(WebCore::MediaRecorderPrivateWriterAVFObjC::appendVideoFrame):
(WebCore::MediaRecorderPrivateWriterAVFObjC::appendAudioSampleBuffer):
(WebCore::MediaRecorderPrivateWriterAVFObjC::finishedFlushingSamples):
(WebCore::MediaRecorderPrivateWriterAVFObjC::stopRecording):
(WebCore::MediaRecorderPrivateWriterAVFObjC::fetchData):
(WebCore::MediaRecorderPrivateWriterAVFObjC::completeFetchData):
(WebCore::MediaRecorderPrivateWriterAVFObjC::appendData):
(WebCore::MediaRecorderPrivateWriterAVFObjC::takeData):
(WebCore::MediaRecorderPrivateWriterAVFObjC::pause):
(WebCore::MediaRecorderPrivateWriterAVFObjC::resume):
(WebCore::MediaRecorderPrivateWriterAVFObjC::mimeType const):
(WebCore::MediaRecorderPrivateWriterAVFObjC::audioBitRate const):
(WebCore::MediaRecorderPrivateWriterAVFObjC::videoBitRate const):
(WebCore::MediaRecorderPrivateWriter::compressedVideoOutputBufferCallback): Deleted.
(WebCore::MediaRecorderPrivateWriter::compressedAudioOutputBufferCallback): Deleted.
(WebCore::MediaRecorderPrivateWriter::MediaRecorderPrivateWriter): Deleted.
(WebCore::MediaRecorderPrivateWriter::~MediaRecorderPrivateWriter): Deleted.
(WebCore::MediaRecorderPrivateWriter::close): Deleted.
(WebCore::MediaRecorderPrivateWriter::initialize): Deleted.
(WebCore::MediaRecorderPrivateWriter::processNewCompressedVideoSampleBuffers): Deleted.
(WebCore::MediaRecorderPrivateWriter::processNewCompressedAudioSampleBuffers): Deleted.
(WebCore::MediaRecorderPrivateWriter::startAssetWriter): Deleted.
(WebCore::MediaRecorderPrivateWriter::appendCompressedAudioSampleBufferIfPossible): Deleted.
(WebCore::MediaRecorderPrivateWriter::appendCompressedVideoSampleBufferIfPossible): Deleted.
(WebCore::MediaRecorderPrivateWriter::appendCompressedVideoSampleBuffer): Deleted.
(WebCore::MediaRecorderPrivateWriter::appendCompressedSampleBuffers): Deleted.
(WebCore::MediaRecorderPrivateWriter::flushCompressedSampleBuffers): Deleted.
(WebCore::MediaRecorderPrivateWriter::appendVideoFrame): Deleted.
(WebCore::MediaRecorderPrivateWriter::appendAudioSampleBuffer): Deleted.
(WebCore::MediaRecorderPrivateWriter::finishedFlushingSamples): Deleted.
(WebCore::MediaRecorderPrivateWriter::stopRecording): Deleted.
(WebCore::MediaRecorderPrivateWriter::fetchData): Deleted.
(WebCore::MediaRecorderPrivateWriter::completeFetchData): Deleted.
(WebCore::MediaRecorderPrivateWriter::appendData): Deleted.
(WebCore::MediaRecorderPrivateWriter::takeData): Deleted.
(WebCore::MediaRecorderPrivateWriter::pause): Deleted.
(WebCore::MediaRecorderPrivateWriter::resume): Deleted.
(WebCore::MediaRecorderPrivateWriter::mimeType const): Deleted.
(WebCore::MediaRecorderPrivateWriter::audioBitRate const): Deleted.
(WebCore::MediaRecorderPrivateWriter::videoBitRate const): Deleted.
* Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterWebM.h: Copied from Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h.
* Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterWebM.mm: Added.
(WebCore::MediaRecorderPrivateWriterWebMDelegate::MediaRecorderPrivateWriterWebMDelegate):
(WebCore::MediaRecorderPrivateWriterWebMDelegate::addAudioTrack):
(WebCore::MediaRecorderPrivateWriterWebMDelegate::addVideoTrack):
(WebCore::MediaRecorderPrivateWriterWebMDelegate::addFrame):
(WebCore::MediaRecorderPrivateWriterWebMDelegate::forceNewClusterOnNextFrame):
(WebCore::MediaRecorderPrivateWriterWebMDelegate::finalize):
(WebCore::MediaRecorderPrivateWriterWebMDelegate::hasAddedFrame const):
(WebCore::MediaRecorderPrivateWriterWebMDelegate::protectedParent const):
(WebCore::MediaRecorderPrivateWriterWebM::create):
(WebCore::mkvCodeIcForMediaVideoCodecId):
(WebCore::codecStringForMediaVideoCodecId):
(WebCore::MediaRecorderPrivateWriterWebM::MediaRecorderPrivateWriterWebM):
(WebCore::MediaRecorderPrivateWriterWebM::~MediaRecorderPrivateWriterWebM):
(WebCore::MediaRecorderPrivateWriterWebM::close):
(WebCore::MediaRecorderPrivateWriterWebM::compressedAudioOutputBufferCallback):
(WebCore::MediaRecorderPrivateWriterWebM::initialize):
(WebCore::MediaRecorderPrivateWriterWebM::enqueueCompressedAudioSampleBuffers):
(WebCore::MediaRecorderPrivateWriterWebM::maybeStartWriting):
(WebCore::MediaRecorderPrivateWriterWebM::appendVideoFrame):
(WebCore::MediaRecorderPrivateWriterWebM::nextVideoFrameTime const):
(WebCore::MediaRecorderPrivateWriterWebM::resumeVideoTime const):
(WebCore::MediaRecorderPrivateWriterWebM::sampleTime const):
(WebCore::MediaRecorderPrivateWriterWebM::encodePendingVideoFrames):
(WebCore::MediaRecorderPrivateWriterWebM::appendAudioSampleBuffer):
(WebCore::MediaRecorderPrivateWriterWebM::flushEncodedQueues):
(WebCore::MediaRecorderPrivateWriterWebM::partiallyFlushEncodedQueues):
(WebCore::MediaRecorderPrivateWriterWebM::muxNextFrame):
(WebCore::MediaRecorderPrivateWriterWebM::stopRecording):
(WebCore::MediaRecorderPrivateWriterWebM::fetchData):
(WebCore::MediaRecorderPrivateWriterWebM::completeFetchData):
(WebCore::MediaRecorderPrivateWriterWebM::flushPendingData):
(WebCore::MediaRecorderPrivateWriterWebM::appendData):
(WebCore::MediaRecorderPrivateWriterWebM::takeData):
(WebCore::MediaRecorderPrivateWriterWebM::pause):
(WebCore::MediaRecorderPrivateWriterWebM::resume):
(WebCore::MediaRecorderPrivateWriterWebM::mimeType const):
(WebCore::MediaRecorderPrivateWriterWebM::audioBitRate const):
(WebCore::MediaRecorderPrivateWriterWebM::videoBitRate const):
(WebCore::MediaRecorderPrivateWriterWebM::maybeForceNewCluster):
* Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.cpp:
(WebKit::RemoteMediaRecorder::fetchData):
* Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.h:
* Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.messages.in:
* Source/WebKit/UIProcess/Cocoa/PlaybackSessionManagerProxy.mm:
(WebKit::PlaybackSessionManagerProxy::seekableRangesVectorChanged):
* Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp:
(WebKit::MediaRecorderPrivate::fetchData):

Canonical link: https://commits.webkit.org/285061@main

To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications