[webkit-changes] [WebKit/WebKit] 7f1bcb: [Cocoa] "Pop" of bad audio heard at the start of c...

[Jer Noble]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 7f1bcb55362b63563bfcb60c4bbb4868c0f310d8
      https://github.com/WebKit/WebKit/commit/7f1bcb55362b63563bfcb60c4bbb4868c0f310d8
  Author: Jer Noble <jer.noble at apple.com>
  Date:   2023-04-11 (Tue, 11 Apr 2023)

  Changed paths:
    M LayoutTests/media/media-source/media-webm-opus-partial-abort-expected.txt
    M LayoutTests/media/media-source/media-webm-opus-partial-abort.html
    M LayoutTests/media/media-source/media-webm-opus-partial-expected.txt
    M LayoutTests/media/media-source/media-webm-opus-partial.html
    M LayoutTests/media/video-test.js
    M Source/WebCore/platform/MediaSample.h
    M Source/WebCore/platform/graphics/cocoa/CMUtilities.mm
    M Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.cpp
    M Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.h

  Log Message:
  -----------
  [Cocoa] "Pop" of bad audio heard at the start of certain YouTube videos
https://bugs.webkit.org/show_bug.cgi?id=255212
rdar://106976225

Reviewed by Eric Carlson.

Tracking addition of a test via https://bugs.webkit.org/show_bug.cgi?id=255227.

Two interrelated problems cause discontinuties in the audio output at the
start of certain Opus-encoded WebM files.

1) A bug in the ffmpeg muxer causes the initial block in a cluster to be 1ms
too long, which causes an audible discontinuity to be generated from
AVSampleBufferAudioRenderer.

2) Some Opus-encoded WebM files include a CodecDelay value, which requires
players to decode, but not render, the initial audio frames in a stream.

For 2), map the CodecDelay value to a kCMSampleBufferAttachmentKey_TrimDurationAtStart
attachment in the resulting CMSampleBuffer. This causes the output duration of the
sample to be reduced by the trim duration, and the output presentation time to be
increased by the trim duration, so also shift the input presentation time by the same
amount. This aligns the first audible frame with the start time of the track.

For 1), if a discontinuity is encountered, and the discontinuity is less than 15ms
simply advance the presentation time of the subsequent sample by the discontinuity
duration. Track this discontinuity cumulatively, so that if multiple discontinuities
are encountered that total greater than 15ms, a real audible discontinuity is generated
and the track is brought back in sync with the master timeline.

* Source/WebCore/platform/MediaSample.h:
* Source/WebCore/platform/graphics/cocoa/CMUtilities.mm:
(WebCore::toCMSampleBuffer):
* Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.cpp:
(WebCore::WebMParser::VideoTrackData::consumeFrameData):
(WebCore::WebMParser::AudioTrackData::AudioTrackData):
(WebCore::WebMParser::AudioTrackData::consumeFrameData):
* Source/WebCore/platform/graphics/cocoa/SourceBufferParserWebM.h:
(WebCore::WebMParser::AudioTrackData::AudioTrackData): Deleted.

Canonical link: https://commits.webkit.org/262837@main