[webkit-changes] [WebKit/WebKit] d9d895: [Cocoa] Improve support for VP codecs in TrackConf...

[Jer Noble]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: d9d895830a49d1c4fa6aaef5ab048c22bf47a176
      https://github.com/WebKit/WebKit/commit/d9d895830a49d1c4fa6aaef5ab048c22bf47a176
  Author: Jer Noble <jer.noble at apple.com>
  Date:   2023-05-10 (Wed, 10 May 2023)

  Changed paths:
    A LayoutTests/media/media-source/content/test-vp9-profile0-manifest.json
    A LayoutTests/media/media-source/content/test-vp9-profile0.webm
    A LayoutTests/media/media-source/content/test-vp9-profile2-manifest.json
    A LayoutTests/media/media-source/content/test-vp9-profile2.webm
    A LayoutTests/media/media-source/media-source-webm-configuration-change-expected.txt
    A LayoutTests/media/media-source/media-source-webm-configuration-change.html
    M LayoutTests/platform/gtk/TestExpectations
    M LayoutTests/platform/mac/TestExpectations
    M LayoutTests/platform/wpe/TestExpectations
    M Source/WebCore/Modules/webcodecs/VideoColorPrimaries.idl
    M Source/WebCore/Modules/webcodecs/VideoMatrixCoefficients.idl
    M Source/WebCore/Modules/webcodecs/VideoTransferCharacteristics.idl
    M Source/WebCore/platform/MediaSample.h
    M Source/WebCore/platform/graphics/AudioTrackPrivate.h
    M Source/WebCore/platform/graphics/PlatformVideoColorPrimaries.h
    M Source/WebCore/platform/graphics/PlatformVideoMatrixCoefficients.h
    M Source/WebCore/platform/graphics/PlatformVideoTransferCharacteristics.h
    M Source/WebCore/platform/graphics/VP9Utilities.cpp
    M Source/WebCore/platform/graphics/VP9Utilities.h
    M Source/WebCore/platform/graphics/VideoTrackPrivate.h
    M Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm
    M Source/WebCore/platform/graphics/cocoa/AudioTrackPrivateWebM.cpp
    M Source/WebCore/platform/graphics/cocoa/AudioTrackPrivateWebM.h
    M Source/WebCore/platform/graphics/cocoa/SourceBufferParser.h
    M Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.cpp
    M Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.h
    M Source/WebCore/platform/graphics/cocoa/VP9UtilitiesCocoa.mm
    M Source/WebCore/platform/graphics/cocoa/VideoTrackPrivateWebM.cpp
    M Source/WebCore/platform/graphics/cocoa/VideoTrackPrivateWebM.h

  Log Message:
  -----------
  [Cocoa] Improve support for VP codecs in TrackConfiguration
https://bugs.webkit.org/show_bug.cgi?id=253789
rdar://106609888

Reviewed by Eric Carlson.

VP8 and VP9 media content are commonly carried in .webm files, which have mostly optional support
for carrying codec configuration data in the media metadata. This means that at the time where track
data is parsed from the WebM file, video or audio configuration data may not yet exist.

Add support for noticing that the format description of a WebM track has "changed" when the first
keyframe is parsed, and add support for notifying the SourceBufferPrivate when this happens.

When these format descriptions change, push the new values up into the previously parsed
TrackPrivates, and use the values to recreate the track's configuration.

* LayoutTests/media/media-source/content/test-vp9-profile0-manifest.json: Added.
* LayoutTests/media/media-source/content/test-vp9-profile0.webm: Added.
* LayoutTests/media/media-source/content/test-vp9-profile2-manifest.json: Added.
* LayoutTests/media/media-source/content/test-vp9-profile2.webm: Added.
* LayoutTests/media/media-source/media-source-webm-configuration-change-expected.txt: Added.
* LayoutTests/media/media-source/media-source-webm-configuration-change.html: Added.
* Source/WebCore/Modules/webcodecs/VideoColorPrimaries.idl:
* Source/WebCore/Modules/webcodecs/VideoMatrixCoefficients.idl:
* Source/WebCore/Modules/webcodecs/VideoTransferCharacteristics.idl:
* Source/WebCore/platform/MediaSample.h:
* Source/WebCore/platform/graphics/AudioTrackPrivate.h:
(WebCore::AudioTrackPrivate::setFormatDescription):
* Source/WebCore/platform/graphics/PlatformVideoColorPrimaries.h:
* Source/WebCore/platform/graphics/PlatformVideoMatrixCoefficients.h:
* Source/WebCore/platform/graphics/PlatformVideoTransferCharacteristics.h:
* Source/WebCore/platform/graphics/VP9Utilities.cpp:
(WebCore::isValidVPProfile):
(WebCore::isValidBitDepth):
(WebCore::isValidRange):
(WebCore::isValidChromaSubsampling):
(WebCore::parseVPCodecParameters):
(WebCore::createVPCodecParametersString):
* Source/WebCore/platform/graphics/VP9Utilities.h:
* Source/WebCore/platform/graphics/VideoTrackPrivate.h:
(WebCore::VideoTrackPrivate::setFormatDescription):
* Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h:
* Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
(WebCore::SourceBufferPrivateAVFObjC::didUpdateFormatDescriptionForTrackId):
(WebCore::SourceBufferPrivateAVFObjC::append):
* Source/WebCore/platform/graphics/cocoa/AudioTrackPrivateWebM.cpp:
(WebCore::AudioTrackPrivateWebM::codec const):
(WebCore::AudioTrackPrivateWebM::sampleRate const):
(WebCore::AudioTrackPrivateWebM::numberOfChannels const):
(WebCore::AudioTrackPrivateWebM::setFormatDescription):
(WebCore::AudioTrackPrivateWebM::updateConfiguration):
* Source/WebCore/platform/graphics/cocoa/AudioTrackPrivateWebM.h:
* Source/WebCore/platform/graphics/cocoa/SourceBufferParser.h:
* Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.cpp:
(WebCore::WebMParser::OnEbml):
(WebCore::WebMParser::VideoTrackData::consumeFrameData):
(WebCore::WebMParser::AudioTrackData::consumeFrameData):
(WebCore::WebMParser::formatDescriptionChangedForTrackData):
(WebCore::SourceBufferParserWebM::formatDescriptionChangedForTrackID):
* Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.h:
(WebCore::WebMParser::Callback::formatDescriptionChangedForTrackID):
* Source/WebCore/platform/graphics/cocoa/VP9UtilitiesCocoa.mm:
(WebCore::createVideoInfoFromVPCodecConfigurationRecord):
* Source/WebCore/platform/graphics/cocoa/VideoTrackPrivateWebM.cpp:
(WebCore::VideoTrackPrivateWebM::setFormatDescription):
(WebCore::VideoTrackPrivateWebM::codec const):
(WebCore::VideoTrackPrivateWebM::width const):
(WebCore::VideoTrackPrivateWebM::height const):
(WebCore::VideoTrackPrivateWebM::colorSpace const):
(WebCore::VideoTrackPrivateWebM::updateConfiguration):
* Source/WebCore/platform/graphics/cocoa/VideoTrackPrivateWebM.h:

Originally-landed-as: 7455323f34a9. rdar://106609888
Canonical link: https://commits.webkit.org/263919@main