[webkit-changes] [WebKit/WebKit] c4679b: Support video/webm; codecs=h264,pcm in MediaRecorder

Tue Feb 4 18:03:15 PST 2025

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: c4679bce9e547c744741633a116a98e8a01464ed
      https://github.com/WebKit/WebKit/commit/c4679bce9e547c744741633a116a98e8a01464ed
  Author: Jean-Yves Avenard <jya at apple.com>
  Date:   2025-02-04 (Tue, 04 Feb 2025)

  Changed paths:
    A LayoutTests/http/wpt/mediarecorder/MediaRecorder-matroska-AV-audio-video-dataavailable-expected.txt
    A LayoutTests/http/wpt/mediarecorder/MediaRecorder-matroska-AV-audio-video-dataavailable.html
    M LayoutTests/platform/glib/TestExpectations
    M Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml
    M Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.cpp
    M Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.h
    M Source/WebCore/SourcesCocoa.txt
    M Source/WebCore/WebCore.xcodeproj/project.pbxproj
    M Source/WebCore/html/HTMLMediaElement.cpp
    M Source/WebCore/html/HTMLMediaElement.h
    M Source/WebCore/page/Quirks.cpp
    M Source/WebCore/page/Quirks.h
    M Source/WebCore/platform/graphics/MIMESniffer.cpp
    M Source/WebCore/platform/graphics/MediaPlayer.cpp
    M Source/WebCore/platform/graphics/MediaPlayer.h
    M Source/WebCore/platform/graphics/MediaPlayerPrivate.h
    M Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp
    M Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm
    M Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm
    M Source/WebCore/platform/graphics/cocoa/AudioTrackPrivateWebM.cpp
    A Source/WebCore/platform/graphics/cocoa/H264UtilitiesCocoa.h
    A Source/WebCore/platform/graphics/cocoa/H264UtilitiesCocoa.mm
    M Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.h
    M Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.mm
    M Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.cpp
    M Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.h
    M Source/WebCore/platform/graphics/cocoa/VideoTrackPrivateWebM.cpp
    M Source/WebCore/platform/graphics/cocoa/WebCoreDecompressionSession.mm
    M Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp
    M Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h
    M Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp
    M Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h
    M Source/WebCore/platform/ios/wak/WebCoreThread.mm
    M Source/WebCore/platform/mac/WebPlaybackControlsManager.mm
    M Source/WebCore/platform/mediarecorder/MediaRecorderPrivateAVFImpl.cpp
    M Source/WebCore/platform/mediarecorder/MediaRecorderPrivateEncoder.cpp
    M Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterWebM.cpp
    M Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.cpp
    M Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.h
    M Source/WebCore/platform/video-codecs/BitReader.cpp
    M Source/WebCore/platform/video-codecs/BitReader.h
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in
    M Source/WebKit/Scripts/webkit/messages.py
    M Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h

  Log Message:
  -----------
  Support video/webm;codecs=h264,pcm in MediaRecorder
https://bugs.webkit.org/show_bug.cgi?id=286409
rdar://143087016

Reviewed by Youenn Fablet.

For web compatibility purposes, we add partial matroska support by allowing h264 and pcm codec in a webm.
This feature is placed behind a pref or if a site has a quirk to require it.

To achieve it we needed the following stages:
- Allow MediaRecorderPrivateWriterWebM accept a H264 and PCM track.
- Allow the SourceBufferParserWebM parse a H264 and PCM track.
  - We add a H264UtilitiesCocoa containing a method to parse the AVCC blob and
    and extract the SPS and PPS NALs to determine the characteristics of the video
    (we ignore the webm's container information in line with how we handle VP8 or VP9.
- Add playback support to the MediaPlayerPrivateWebM to be able to play the videos the MediaRecorder created.
- Add a MediaPlayer::LoadingOptions object to the MediaPlayer and MediaPlayerPrivate so that
  we can activate limited matroska support on a per site/video basis.
  LoadingOptions will also allow easier flexibility to serialise over IPC between the content and GPU process.
- Place the entire feature behind a developer pref (off by default) which can be overridden by a quirk.

* LayoutTests/http/wpt/mediarecorder/MediaRecorder-matroska-AV-audio-video-dataavailable-expected.txt: Added.
* LayoutTests/http/wpt/mediarecorder/MediaRecorder-matroska-AV-audio-video-dataavailable.html: Added.
* LayoutTests/platform/glib/TestExpectations:
* Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml:
* Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.cpp:
* Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.h:
* Source/WebCore/SourcesCocoa.txt:
* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::canPlayType const):
(WebCore::HTMLMediaElement::loadResource):
(WebCore::HTMLMediaElement::setNetworkState):
(WebCore::HTMLMediaElement::selectNextSourceChild):
* Source/WebCore/page/Quirks.cpp:
(WebCore::Quirks::needsLimitedMatroskaSupport const):
* Source/WebCore/page/Quirks.h:
* Source/WebCore/platform/graphics/MIMESniffer.cpp:
(WebCore::MIMESniffer::hasSignatureForWebM):
* Source/WebCore/platform/graphics/MediaPlayer.cpp:
(WebCore::MediaPlayer::load):
(WebCore::MediaPlayer::nextBestMediaEngine):
(WebCore::MediaPlayer::loadWithNextMediaEngine):
(WebCore::MediaPlayer::contentMIMEType const):
(WebCore::MediaPlayer::contentTypeCodecs const):
(WebCore::MediaPlayer::contentMIMETypeWasInferredFromExtension const):
* Source/WebCore/platform/graphics/MediaPlayer.h:
* Source/WebCore/platform/graphics/MediaPlayerPrivate.h:
(WebCore::MediaPlayerPrivateInterface::load):
* Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp:
(WebCore::MediaPlayerPrivateAVFoundation::load):
* Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h:
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::supportsLimitedMatroska const):
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::load):
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::load):
* Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
(WebCore::m_logIdentifier):
* Source/WebCore/platform/graphics/cocoa/AudioTrackPrivateWebM.cpp:
(WebCore::AudioTrackPrivateWebM::codec const):
* Source/WebCore/platform/graphics/cocoa/H264UtilitiesCocoa.mm: Added.
(WebCore::createVideoInfoFromAVCC):
* Source/WebCore/platform/graphics/cocoa/H264UtilitiesCocoa.h:
* Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.h:
* Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.mm:
(WebCore::MediaPlayerPrivateWebM::supportsType):
(WebCore::MediaPlayerPrivateWebM::load):
* Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.cpp:
(WebCore::WebMParser::OnEbml):
(WebCore::WebMParser::OnTrackEntry):
(WebCore::WebMParser::OnFrame):
(WebCore::WebMParser::VideoTrackData::consumeFrameData):
(WebCore::WebMParser::AudioTrackData::consumeFrameData):
(WebCore::WebMParser::isSupportedVideoCodec):
(WebCore::WebMParser::isSupportedAudioCodec):
(WebCore::SourceBufferParserWebM::isContentTypeSupported):
* Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.h:
(WebCore::WebMParser::TrackData::consumeFrameData):
(WebCore::WebMParser::allowLimitedMatroska):
(WebCore::SourceBufferParserWebM::allowLimitedMatroska):
* Source/WebCore/platform/graphics/cocoa/VideoTrackPrivateWebM.cpp:
(WebCore::VideoTrackPrivateWebM::codec const):
* Source/WebCore/platform/graphics/cocoa/WebCoreDecompressionSession.mm: Fixing unified build due to missing header.
* Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
(WebCore::MediaPlayerPrivateGStreamer::load):
* Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp:
(WebCore::MediaPlayerPrivateGStreamerMSE::load):
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h:
* Source/WebCore/platform/ios/wak/WebCoreThread.mm:
* Source/WebCore/platform/mac/WebPlaybackControlsManager.mm: Fix unified build.
* Source/WebCore/platform/mediarecorder/MediaRecorderPrivateAVFImpl.cpp:
(WebCore::MediaRecorderPrivateAVFImpl::isTypeSupported):
* Source/WebCore/platform/mediarecorder/MediaRecorderPrivateEncoder.cpp:
(WebCore::codecStringForMediaVideoCodecId):
(WebCore::MediaRecorderPrivateEncoder::initialize):
(WebCore::MediaRecorderPrivateEncoder::audioSamplesDescriptionChanged):
(WebCore::MediaRecorderPrivateEncoder::enqueueCompressedAudioSampleBuffers):
* Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterWebM.cpp:
(WebCore::mkvCodeIcForMediaVideoCodecId):
(WebCore::MediaRecorderPrivateWriterWebMDelegate::addAudioTrack):
(WebCore::MediaRecorderPrivateWriterWebMDelegate::addVideoTrack):
* Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.cpp:
(WebCore::MockMediaPlayerMediaSource::load):
* Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.h:
* Source/WebCore/platform/video-codecs/BitReader.cpp:
(WebCore::BitReader::readBit):
(WebCore::BitReader::skipBytes):
(WebCore::BitReader::bitOffset const):
* Source/WebCore/platform/video-codecs/BitReader.h:
(WebCore::BitReader::read):
(WebCore::BitReader::byteOffset const):
(WebCore::BitReader::readBytes):
(WebKit::RemoteMediaPlayerProxy::load):
(WebKit::RemoteMediaPlayerProxy::loadMediaSource):
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h:
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
* Source/WebKit/GPUProcess/media/RemoteTrackInfo.h:
(WebKit::RemoteAudioInfo::toAudioInfo const):
* Source/WebKit/Scripts/webkit/messages.py:
(headers_for_type):
* Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in:
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
(WebKit::MediaPlayerPrivateRemote::load):
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h:

Canonical link: https://commits.webkit.org/289843@main

To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications