[webkit-changes] [WebKit/WebKit] 1fc26a: Terrible video quality when using TransformStream ...

[youennf]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 1fc26a73bcc7fe0b58abd15422048e7023650dcc
      https://github.com/WebKit/WebKit/commit/1fc26a73bcc7fe0b58abd15422048e7023650dcc
  Author: Youenn Fablet <youennf at gmail.com>
  Date:   2023-10-05 (Thu, 05 Oct 2023)

  Changed paths:
    A LayoutTests/http/wpt/webrtc/video-script-transform-simulcast-expected.txt
    A LayoutTests/http/wpt/webrtc/video-script-transform-simulcast.html
    M LayoutTests/platform/glib/TestExpectations
    M Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpTransformBackend.cpp
    M Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpTransformBackend.h

  Log Message:
  -----------
  Terrible video quality when using TransformStream with Simulcast
https://bugs.webkit.org/show_bug.cgi?id=257803
rdar://110395571

Reviewed by Jean-Yves Avenard.

The libwebrtc backend provides one callback per ssrc generated by a sender.
In non simulcast cases, there is only one callback.
In simulcast case, there is one callback per simulcast encoding.
Before the patch, we would use one callback for all video encoded frames, thus sending all simulcast streams in the same ssrc.
This would confuse receivers and would lead to decoding failures, leading to sending PLI/FIR, thus leading to bad video quality.

We are now storing these callbacks in a map, and use the callback associated to the ssrc of the frame.
This allows to correctly send each simulcast stream with its associated ssrc.

Covered by added test case.

* LayoutTests/http/wpt/webrtc/video-script-transform-simulcast-expected.txt: Added.
* LayoutTests/http/wpt/webrtc/video-script-transform-simulcast.html: Added.
* LayoutTests/platform/glib/TestExpectations:
* Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpTransformBackend.cpp:
(WebCore::LibWebRTCRtpTransformBackend::addOutputCallback):
(WebCore::LibWebRTCRtpTransformBackend::removeOutputCallback):
(WebCore::LibWebRTCRtpTransformBackend::sendFrameToOutput):
(WebCore::LibWebRTCRtpTransformBackend::processTransformedFrame):
(WebCore::LibWebRTCRtpTransformBackend::Transform):
(WebCore::LibWebRTCRtpTransformBackend::RegisterTransformedFrameCallback):
(WebCore::LibWebRTCRtpTransformBackend::RegisterTransformedFrameSinkCallback):
(WebCore::LibWebRTCRtpTransformBackend::UnregisterTransformedFrameCallback):
(WebCore::LibWebRTCRtpTransformBackend::UnregisterTransformedFrameSinkCallback):
(WebCore::LibWebRTCRtpTransformBackend::setOutputCallback): Deleted.
* Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpTransformBackend.h:

Canonical link: https://commits.webkit.org/268912@main