[webkit-changes] [WebKit/WebKit] f2328d: Add WebCodecs H264 L1T2 support

[youennf]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: f2328d7f963e79478b5d63a5141478b81b895ac9
      https://github.com/WebKit/WebKit/commit/f2328d7f963e79478b5d63a5141478b81b895ac9
  Author: Youenn Fablet <youennf at gmail.com>
  Date:   2023-10-13 (Fri, 13 Oct 2023)

  Changed paths:
    M LayoutTests/imported/w3c/web-platform-tests/webcodecs/temporal-svc-encoding.https.any.js
    M LayoutTests/imported/w3c/web-platform-tests/webcodecs/temporal-svc-encoding.https.any.worker_h264-expected.txt
    M LayoutTests/imported/w3c/web-platform-tests/webcodecs/temporal-svc-encoding.https.any_h264-expected.txt
    M LayoutTests/platform/ios-simulator/TestExpectations
    M LayoutTests/platform/mac-wk2/TestExpectations
    M Source/ThirdParty/libwebrtc/Configurations/libwebrtc.exp
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.h
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.mm
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/base/RTCEncodedImage.h
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.h
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.mm
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH265.mm
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/helpers.cc
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/helpers.h
    M Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h
    M Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in
    M Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm
    M Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in
    M Source/WebKit/WebProcess/GPU/media/RemoteVideoCodecFactory.cpp
    M Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp
    M Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.h

  Log Message:
  -----------
  Add WebCodecs H264 L1T2 support
https://bugs.webkit.org/show_bug.cgi?id=261118
rdar://114940765

Reviewed by Jean-Yves Avenard.

Allow L1T2 for H264 in RemoteVideoCodecFactory::createEncoder.

Add support for L1T2 in RTCVideoEncoderH264.
We make use of kCMSampleAttachmentKey_IsDependedOnByOthers and kVTCompressionPropertyKey_BaseLayerFrameRateFraction to implement it.
We update the code path to pass to GPUProcess the scalability mode and from GPUProcess the temporalndex.

We also remove DataRateLimits usage in H264 and H265 encoder code paths.
This is not really useful for the low latency code path since we are providing the average bit rate and we are using kVTVideoEncoderSpecification_RequiredLowLatency.

* LayoutTests/imported/w3c/web-platform-tests/webcodecs/temporal-svc-encoding.https.any.js:
(promise_setup):
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/temporal-svc-encoding.https.any.worker_h264-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/webcodecs/temporal-svc-encoding.https.any_h264-expected.txt:
* LayoutTests/platform/ios-simulator/TestExpectations:
* LayoutTests/platform/mac-wk2/TestExpectations:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.exp:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.h:
(webrtc::WebKitEncodedFrameInfo::decode):
(webrtc::WebKitEncodedFrameInfo::encode const):
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.mm:
(-[WK_RTCLocalVideoH264H265Encoder initWithCodecInfo:scalabilityMode:]):
(webrtc::createLocalEncoder):
(-[WK_RTCLocalVideoH264H265Encoder initWithCodecInfo:]): Deleted.
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/base/RTCEncodedImage.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.mm:
(-[RTCVideoEncoderH264 initWithCodecInfo:]):
(-[RTCVideoEncoderH264 configureCompressionSession]):
(-[RTCVideoEncoderH264 setEncoderBitrateBps:frameRate:]):
(-[RTCVideoEncoderH264 frameWasEncoded:flags:sampleBuffer:codecSpecificInfo:width:height:renderTimeMs:timestamp:duration:rotation:isKeyFrameRequired:]):
(-[RTCVideoEncoderH264 enableL1T2ScalabilityMode]):
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH265.mm:
(-[RTCVideoEncoderH265 setEncoderBitrateBps:]):
(-[RTCVideoEncoderH265 frameWasEncoded:flags:sampleBuffer:width:height:renderTimeMs:timestamp:rotation:]):
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/helpers.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/helpers.h:
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h:
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in:
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm:
(WebKit::validateEncoderConfiguration):
(WebKit::LibWebRTCCodecsProxy::createEncoder):
(WebKit::validateEncoderCodecString): Deleted.
* Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in:
* Source/WebKit/WebProcess/GPU/media/RemoteVideoCodecFactory.cpp:
(WebKit::RemoteVideoCodecFactory::createEncoder):
(WebKit::RemoteVideoEncoder::RemoteVideoEncoder):
(WebKit::RemoteVideoEncoderCallbacks::notifyEncodedChunk):
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp:
(WebKit::LibWebRTCCodecs::createEncoder):
(WebKit::LibWebRTCCodecs::createEncoderAndWaitUntilInitialized):
(WebKit::LibWebRTCCodecs::createEncoderInternal):
(WebKit::LibWebRTCCodecs::completedEncoding):
(WebKit::LibWebRTCCodecs::gpuProcessConnectionDidClose):
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.h:

Canonical link: https://commits.webkit.org/269297@main