[webkit-changes] [WebKit/WebKit] 9d2580: WebCodecs encoders should be able to encode synchr...

[youennf]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 9d258046f92b2ce5a295f7fb6a6a6336039a8acc
      https://github.com/WebKit/WebKit/commit/9d258046f92b2ce5a295f7fb6a6a6336039a8acc
  Author: Youenn Fablet <youennf at gmail.com>
  Date:   2023-06-28 (Wed, 28 Jun 2023)

  Changed paths:
    A LayoutTests/http/wpt/webcodecs/videoFrame-duration-expected.txt
    A LayoutTests/http/wpt/webcodecs/videoFrame-duration.html
    M Source/ThirdParty/libwebrtc/Configurations/libwebrtc.exp
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.h
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.mm
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/base/RTCEncodedImage.h
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/base/RTCEncodedImage.m
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/base/RTCVideoFrame.h
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/base/RTCVideoFrame.mm
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.mm
    M Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h
    M Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in
    M Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm
    M Source/WebKit/WebProcess/GPU/media/RemoteVideoCodecFactory.cpp
    M Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp
    M Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.h

  Log Message:
  -----------
  WebCodecs encoders should be able to encode synchronously two frames with the same timestamp
https://bugs.webkit.org/show_bug.cgi?id=258568
rdar://111389746

Reviewed by Eric Carlson.

We are using a map to retrieve frame durations on WebProcess side based on timestamp.
This does not work if timestamps are the same for two frames that are being encoded.
Instead of doing that, we are now sending both timestamp and duration to GPUProcess when encoding a frame.
GPUProcess sends back both information as well, which allows to preserve them.

This patch is also fixing negative timestamps as they were cast to uint32_t in GPUProcess.
Duration being optional, we are using std::numeric_limits<uint64_t>::max() as a missing duration.

* LayoutTests/http/wpt/webcodecs/videoFrame-duration-expected.txt: Added.
* LayoutTests/http/wpt/webcodecs/videoFrame-duration.html: Added.
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.exp:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.h:
(webrtc::WebKitEncodedFrameInfo::decode):
(webrtc::WebKitEncodedFrameInfo::encode const):
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.mm:
(webrtc::createLocalEncoder):
(webrtc::encodeLocalEncoderFrame):
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/base/RTCEncodedImage.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/base/RTCEncodedImage.m:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/base/RTCVideoFrame.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/base/RTCVideoFrame.mm:
(-[RTCVideoFrame duration]):
(-[RTCVideoFrame initWithBuffer:rotation:timeStampNs:]):
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.mm:
(-[RTCVideoEncoderH264 encode:codecSpecificInfo:frameTypes:]):
(-[RTCVideoEncoderH264 frameWasEncoded:flags:sampleBuffer:codecSpecificInfo:width:height:renderTimeMs:timestamp:duration:rotation:isKeyFrameRequired:]):
(-[RTCVideoEncoderH264 frameWasEncoded:flags:sampleBuffer:codecSpecificInfo:width:height:renderTimeMs:timestamp:rotation:isKeyFrameRequired:]): Deleted.
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h:
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in:
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm:
(WebKit::LibWebRTCCodecsProxy::encodeFrame):
* Source/WebKit/WebProcess/GPU/media/RemoteVideoCodecFactory.cpp:
(WebKit::RemoteVideoEncoderCallbacks::close):
(WebKit::RemoteVideoEncoder::RemoteVideoEncoder):
(WebKit::RemoteVideoEncoder::encode):
(WebKit::RemoteVideoEncoderCallbacks::notifyEncodedChunk):
(WebKit::RemoteVideoEncoderCallbacks::addDuration): Deleted.
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp:
(WebKit::LibWebRTCCodecs::encodeFrameInternal):
(WebKit::LibWebRTCCodecs::encodeFrame):
(WebKit::LibWebRTCCodecs::completedEncoding):
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.h:

Canonical link: https://commits.webkit.org/265580@main