[webkit-changes] [WebKit/WebKit] 4366e9: MediaRecorderPrivateEncoder can write frames out o...

[Jean-Yves Avenard]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 4366e9326ff6f41042ca4b61d99e32d1105b1c53
      https://github.com/WebKit/WebKit/commit/4366e9326ff6f41042ca4b61d99e32d1105b1c53
  Author: Jean-Yves Avenard <jya at apple.com>
  Date:   2025-01-31 (Fri, 31 Jan 2025)

  Changed paths:
    M LayoutTests/http/wpt/mediarecorder/MediaRecorder-AV-audio-video-dataavailable.html
    A LayoutTests/http/wpt/mediarecorder/MediaRecorder-webm-AV-audio-video-dataavailable-expected.txt
    A LayoutTests/http/wpt/mediarecorder/MediaRecorder-webm-AV-audio-video-dataavailable.html
    M Source/WebCore/platform/mediarecorder/MediaRecorderPrivateEncoder.cpp
    M Source/WebCore/platform/mediarecorder/MediaRecorderPrivateEncoder.h

  Log Message:
  -----------
  MediaRecorderPrivateEncoder can write frames out of order
https://bugs.webkit.org/show_bug.cgi?id=286805
rdar://143956063

Reviewed by Youenn Fablet.

When we write to the container the video frame, we must first ensure that
we have written any earlier audio frames as WebM requires monotonically increasing timestamps.
To do so, the code could wait on in-flight audio frames prior writing the video.
However, if we paused or stopped the recorder, the waiting promise would be
immediately rejected which caused the video frame to be immediately written
only to drain/finish the audio converter which could retrieve audio frames
with a timestamp lesser than the video written.
This used to cause an assertion which was changed to a release log error instead in
287869 at main.

Another issue was that it was assume we always had an audio frame with the same
presentation time as the latest video frame (as the audio clock drives the time)
However, if two video frames arrive in a burst before new audio is received
and to avoid having the video frames with the same timestamp we add 1us.
This could cause the wait for pending audio to never ends as this amended.

We can now remove some of the debug assertions we added in 287869 at main and
re-introduce the ASSERT.

Added a test that exercising the code by creating a MediaRecorder with a short
timeslice.
* LayoutTests/http/wpt/mediarecorder/MediaRecorder-AV-audio-video-dataavailable.html: Make oscillator variable global to get around webkit.org/b/271227
* LayoutTests/http/wpt/mediarecorder/MediaRecorder-webm-AV-audio-video-dataavailable-expected.txt: Added.
* LayoutTests/http/wpt/mediarecorder/MediaRecorder-webm-AV-audio-video-dataavailable.html: Added.
* Source/WebCore/platform/mediarecorder/MediaRecorderPrivateEncoder.cpp:
(WebCore::MediaRecorderPrivateEncoder::pause):
(WebCore::MediaRecorderPrivateEncoder::appendVideoFrame):
(WebCore::MediaRecorderPrivateEncoder::enqueueCompressedVideoFrame):
(WebCore::MediaRecorderPrivateEncoder::waitForMatchingAudio):
(WebCore::MediaRecorderPrivateEncoder::interleaveAndEnqueueNextFrame):
(WebCore::MediaRecorderPrivateEncoder::stopRecording):
(WebCore::MediaRecorderPrivateEncoder::flushPendingData):
* Source/WebCore/platform/mediarecorder/MediaRecorderPrivateEncoder.h:

Canonical link: https://commits.webkit.org/289643@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications