[webkit-changes] [WebKit/WebKit] 279e75: [Cocoa] Remote video codecs should surface any err...

[youennf]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 279e75f6851c27fc51dc6ed4e36641f5d354bc40
      https://github.com/WebKit/WebKit/commit/279e75f6851c27fc51dc6ed4e36641f5d354bc40
  Author: Youenn Fablet <youennf at gmail.com>
  Date:   2023-12-01 (Fri, 01 Dec 2023)

  Changed paths:
    M LayoutTests/TestExpectations
    A LayoutTests/http/wpt/webcodecs/encoder-task-failing-expected.txt
    A LayoutTests/http/wpt/webcodecs/encoder-task-failing.html
    M LayoutTests/platform/mac-wk2/TestExpectations
    M Source/ThirdParty/libwebrtc/Configurations/libwebrtc.exp
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.h
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.mm
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/base/RTCVideoEncoder.h
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.h
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.mm
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH265.h
    M Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH265.mm
    M Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h
    M Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in
    M Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm
    M Source/WebKit/WebProcess/GPU/media/RemoteVideoCodecFactory.cpp
    M Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp
    M Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.h
    M Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.messages.in

  Log Message:
  -----------
  [Cocoa] Remote video codecs should surface any error happening during encoding to web pages
https://bugs.webkit.org/show_bug.cgi?id=265307
rdar://118770776

Reviewed by Eric Carlson.

We were not exposing the encoder errors to JavaScript, while the spec asks us to do so.
We add to H264 and HEVC encoders a way to be notified of any error, and update the code so that,
either the success or error callback is called for every encoding task.

We update LibWebRTCCodecs to make encoding and flush IPC messages with a response handler.
This allows to call the WebCodecs encoding task completion handler with the result of the encoding task.
We use NativePromise for both as a convenience and we make sure that the order of messages is guaranteed.

Covered by a new test which triggers an encoding error by using a low H264 encoding level with large video frames.
This should in theory be handled at configure step but we are not yet checking precisely enough this.
When we do this, we will need to change the test and it will probably be pass for all platforms as the checking code might be in WebKit and not below.
This test is only passing in Sonoma since below, the underlying encoder is not triggering an encoding error.

* LayoutTests/TestExpectations:
* LayoutTests/http/wpt/webcodecs/encoder-task-failing-expected.txt: Added.
* LayoutTests/http/wpt/webcodecs/encoder-task-failing.html: Added.
* LayoutTests/platform/mac-wk2/TestExpectations:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.exp:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.mm:
(-[WK_RTCLocalVideoH264H265Encoder setErrorCallback:]):
(webrtc::createLocalEncoder):
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/base/RTCVideoEncoder.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.mm:
(-[RTCVideoEncoderH264 setErrorCallback:]):
(-[RTCVideoEncoderH264 encode:codecSpecificInfo:frameTypes:]):
(-[RTCVideoEncoderH264 frameWasEncoded:flags:sampleBuffer:codecSpecificInfo:width:height:renderTimeMs:timestamp:duration:rotation:isKeyFrameRequired:]):
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH265.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH265.mm:
(-[RTCVideoEncoderH265 setErrorCallback:]):
(-[RTCVideoEncoderH265 encode:codecSpecificInfo:frameTypes:]):
(-[RTCVideoEncoderH265 frameWasEncoded:flags:sampleBuffer:width:height:renderTimeMs:timestamp:rotation:]):
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h:
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in:
* Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm:
(WebKit::LibWebRTCCodecsProxy::createEncoder):
(WebKit::LibWebRTCCodecsProxy::encodeFrame):
(WebKit::LibWebRTCCodecsProxy::notifyEncoderResult):
(WebKit::LibWebRTCCodecsProxy::flushEncoder):
* Source/WebKit/WebProcess/GPU/media/RemoteVideoCodecFactory.cpp:
(WebKit::RemoteVideoEncoderCallbacks::close):
(WebKit::RemoteVideoEncoder::encode):
(WebKit::RemoteVideoEncoderCallbacks::notifyEncodedChunk):
(WebKit::RemoteVideoEncoderCallbacks::addDuration): Deleted.
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp:
(WebKit::LibWebRTCCodecs::encodeFrameInternal):
(WebKit::LibWebRTCCodecs::encodeFrame):
(WebKit::LibWebRTCCodecs::flushEncoder):
(WebKit::LibWebRTCCodecs::flushEncoderCompleted): Deleted.
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.h:
* Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.messages.in:

Canonical link: https://commits.webkit.org/271377@main