[webkit-changes] [WebKit/WebKit] ee5de7: REGRESSION(262173 at main): media/media-source/media-...

Jean-Yves Avenard noreply at github.com
Fri Aug 25 07:17:58 PDT 2023 

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: ee5de7a0c1f24d88e04b8b4dff25dccfa14faa0f
      https://github.com/WebKit/WebKit/commit/ee5de7a0c1f24d88e04b8b4dff25dccfa14faa0f
  Author: Jean-Yves Avenard <jya at apple.com>
  Date:   2023-08-25 (Fri, 25 Aug 2023)

  Changed paths:
    M LayoutTests/platform/mac-wk2/TestExpectations
    M Source/WebCore/Modules/mediasource/MediaSource.cpp
    M Source/WebCore/Modules/mediasource/MediaSource.h
    M Source/WebCore/Modules/mediasource/SourceBuffer.cpp
    M Source/WebCore/Modules/mediasource/SourceBuffer.h
    M Source/WebCore/html/HTMLMediaElement.cpp
    M Source/WebCore/html/HTMLMediaElement.h
    M Source/WebCore/platform/graphics/MediaPlayer.cpp
    M Source/WebCore/platform/graphics/MediaPlayer.h
    M Source/WebCore/platform/graphics/MediaPlayerPrivate.h
    M Source/WebCore/platform/graphics/MediaSourcePrivate.h
    M Source/WebCore/platform/graphics/MediaSourcePrivateClient.h
    M Source/WebCore/platform/graphics/SourceBufferPrivate.cpp
    M Source/WebCore/platform/graphics/SourceBufferPrivate.h
    M Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp
    M Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.mm
    M Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm
    M Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.h
    M Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.mm
    M Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp
    M Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h
    M Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp
    M Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h
    M Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.cpp
    M Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.h
    M Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.cpp
    M Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.h
    M Source/WebCore/platform/graphics/win/MediaPlayerPrivateMediaFoundation.cpp
    M Source/WebCore/platform/graphics/win/MediaPlayerPrivateMediaFoundation.h
    M Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.cpp
    M Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.h
    M Source/WebCore/platform/mock/mediasource/MockMediaSourcePrivate.cpp
    M Source/WebCore/platform/mock/mediasource/MockMediaSourcePrivate.h
    M Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.cpp
    M Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.h
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in
    M Source/WebKit/GPUProcess/media/RemoteMediaSourceProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteMediaSourceProxy.h
    M Source/WebKit/GPUProcess/media/RemoteMediaSourceProxy.messages.in
    M Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.h
    M Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.messages.in
    M Source/WebKit/Scripts/webkit/messages.py
    M Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h
    M Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.h
    M Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.messages.in
    M Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerState.h
    M Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerState.serialization.in
    M Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.h

  Log Message:
  -----------
  REGRESSION(262173 at main): media/media-source/media-source-fastseek.html is a flaky text failure
https://bugs.webkit.org/show_bug.cgi?id=260185
rdar://113881009

Reviewed by Youenn Fablet.

fastSeek and normal Seek followed different paths and the flow between
GPU process and content process was disconnected. This leads to a racy
behaviour where the HTMLMediaElement in the content process could assume
that the seek operation in the GPU process had completed when it hadn't.
There's been a few workarounds put in place for this problem but none
fixed the core issue: we can only complete the seek operation once the
media source has received all the data and we must wait rather than
expect the player to keep retrying.

fastSeek time target calculation was also left to each of the MediaPlayerPrivate
to handle.
This change move part of the seek operation back to the content process,
where the MediaSource element will ensure that there is data to continue
the seek.
It will also handle the fastSeek calculation.
Once the MediaSource has seeked each of the SourceBuffer and has determined
the final time destination, a CompletionHandler will be resolved with the
exact to seeked to location. From that point, each MediaPlayerPrivate can
resume the seek operation and once complete notify the HTMLMediaElement so
that it can fire the appropriate events.

This also gets us closer to having the handling of the SourceBuffer's content
back to the content process, so that all unsafe demuxing operations can
be performed there.

In this change, the various seek methods (from float, double, MediaTime),
accurate and fast are combined into a single seekToTarget with a new
SeekTarget object that can be used for all types.
We remove the need for SourceBufferPrivate and MediaSourcePrivate's having
to track if it is currently seeking or not and the need for
waitForSeekCompleted/willSeek/seekCompleted calls.

Platform specific changes:
- MediaPlayerPrivateMediaSourceAVFObjC: the seeking internal logic was broken.
m_synchronizerSeeking which is supposed to monitor the AVSampleBufferRenderSynchronizer
seeking state was always set to false. As such, we would fire the timeupdate/seeked event even
before the proper frame was displayed and the new time was set.
This likely explains several intermittent failures with video reftest.
- MediaPlayerPrivateGStreamerMSE: The structure of the code doesn’t permit an easy migration
to the new asynchronous flow. This is tracked in bug 260607 to be done at a later stage.

Covered by existing tests.

* LayoutTests/platform/mac-wk2/TestExpectations:
* Source/WebCore/Modules/mediasource/MediaSource.cpp:
(WebCore::MediaSource::MediaSource):
(WebCore::MediaSource::currentTime const):
(WebCore::MediaSource::seekToTarget):
(WebCore::MediaSource::completeSeek):
(WebCore::MediaSource::monitorSourceBuffers):
(WebCore::MediaSource::detachFromElement):
(WebCore::MediaSource::seekToTime): Deleted.
* Source/WebCore/Modules/mediasource/MediaSource.h:
(WebCore::MediaSource::isSeeking const):
* Source/WebCore/Modules/mediasource/SourceBuffer.cpp:
(WebCore::SourceBuffer::seekToTarget):
(WebCore::SourceBuffer::seekToTime): Deleted.
* Source/WebCore/Modules/mediasource/SourceBuffer.h:
* Source/WebCore/html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::setReadyState):
(WebCore::HTMLMediaElement::fastSeek):
(WebCore::HTMLMediaElement::seek):
(WebCore::HTMLMediaElement::seekInternal):
(WebCore::HTMLMediaElement::seekWithTolerance):
(WebCore::HTMLMediaElement::seekTask):
(WebCore::HTMLMediaElement::setCurrentTimeWithTolerance):
* Source/WebCore/html/HTMLMediaElement.h:
(WebCore::HTMLMediaElement::PendingSeek::PendingSeek):
* Source/WebCore/platform/graphics/MediaPlayer.cpp:
(WebCore::MediaPlayer::seekToTarget):
(WebCore::MediaPlayer::seekToTime):
(WebCore::MediaPlayer::seekWhenPossible):
(WebCore::MediaPlayer::readyStateChanged):
(WebCore::SeekTarget::toString const):
(WebCore::MediaPlayer::seekWithTolerance): Deleted.
(WebCore::MediaPlayer::seek): Deleted.
* Source/WebCore/platform/graphics/MediaPlayer.h:
(WebCore::SeekTarget::SeekTarget):
(WebCore::SeekTarget::zero):
(WTF::LogArgument<WebCore::SeekTarget>::toString):
* Source/WebCore/platform/graphics/MediaPlayerPrivate.h:
(WebCore::MediaPlayerPrivateInterface::hasClosedCaptions const):
(WebCore::MediaPlayerPrivateInterface::seek): Deleted.
(WebCore::MediaPlayerPrivateInterface::seekDouble): Deleted.
(WebCore::MediaPlayerPrivateInterface::seekWithTolerance): Deleted.
* Source/WebCore/platform/graphics/MediaSourcePrivate.h:
(WebCore::MediaSourcePrivate::setIsSeeking): Deleted.
(WebCore::MediaSourcePrivate::isSeeking const): Deleted.
* Source/WebCore/platform/graphics/MediaSourcePrivateClient.h:
* Source/WebCore/platform/graphics/SourceBufferPrivate.cpp:
(WebCore::SourceBufferPrivate::seekToTarget):
(WebCore::SourceBufferPrivate::fastSeekTimeForMediaTime): Deleted.
* Source/WebCore/platform/graphics/SourceBufferPrivate.h:
(WebCore::SourceBufferPrivate::setMaximumQueueDepthForTrackID):
* Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp:
(WebCore::MediaPlayerPrivateAVFoundation::seekToTarget):
(WebCore::MediaPlayerPrivateAVFoundation::seek): Deleted.
(WebCore::MediaPlayerPrivateAVFoundation::seekWithTolerance): Deleted.
* Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
(WebCore::MediaPlayerPrivateAVFoundationObjC::seekToTargetInternal):
(WebCore::MediaPlayerPrivateAVFoundationObjC::seekToTime): Deleted.
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h:
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::PendingSeek::PendingSeek): Deleted.
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:
(WebCore::convertEnumerationToString):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::MediaPlayerPrivateMediaSourceAVFObjC):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::seekToTarget):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::seekInternal):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::seekCompleted):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::seeking const):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setHasAvailableVideoFrame):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::seekWithTolerance): Deleted.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::waitForSeekCompleted): Deleted.
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.mm:
(WebCore::MediaSourcePrivateAVFObjC::seekToTarget):
(WebCore::MediaSourcePrivateAVFObjC::waitForSeekCompleted): Deleted.
(WebCore::MediaSourcePrivateAVFObjC::seekCompleted): Deleted.
(WebCore::MediaSourcePrivateAVFObjC::seekToTime): Deleted.
(WebCore::MediaSourcePrivateAVFObjC::fastSeekTimeForMediaTime): Deleted.
* Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
* Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.h:
* Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.mm:
(WebCore::MediaPlayerPrivateWebM::play):
(WebCore::MediaPlayerPrivateWebM::seekToTarget):
(WebCore::MediaPlayerPrivateWebM::seek): Deleted.
* Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
(WebCore::MediaPlayerPrivateGStreamer::MediaPlayerPrivateGStreamer):
(WebCore::MediaPlayerPrivateGStreamer::play):
(WebCore::MediaPlayerPrivateGStreamer::doSeek):
(WebCore::MediaPlayerPrivateGStreamer::seekToTarget):
(WebCore::MediaPlayerPrivateGStreamer::updatePlaybackRate):
(WebCore::MediaPlayerPrivateGStreamer::currentMediaTime const):
(WebCore::MediaPlayerPrivateGStreamer::playbackPosition const):
(WebCore::MediaPlayerPrivateGStreamer::finishSeek):
(WebCore::MediaPlayerPrivateGStreamer::updateStates):
(WebCore::MediaPlayerPrivateGStreamer::seek): Deleted.
* Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp:
(WebCore::MediaPlayerPrivateGStreamerMSE::seekToTarget):
(WebCore::MediaPlayerPrivateGStreamerMSE::doSeek):
(WebCore::MediaPlayerPrivateGStreamerMSE::seek): Deleted.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h:
* Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.cpp:
(WebCore::MediaSourcePrivateGStreamer::seekToTarget):
(WebCore::MediaSourcePrivateGStreamer::seekCompleted): Deleted.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.h:
* Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.cpp:
(WebCore::SourceBufferPrivateGStreamer::isSeeking const): Deleted.
* Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.h:
* Source/WebCore/platform/graphics/win/MediaPlayerPrivateMediaFoundation.cpp:
(WebCore::MediaPlayerPrivateMediaFoundation::seekToTarget):
(WebCore::MediaPlayerPrivateMediaFoundation::seek): Deleted.
* Source/WebCore/platform/graphics/win/MediaPlayerPrivateMediaFoundation.h:
* Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.cpp:
(WebCore::MockMediaPlayerMediaSource::seekToTarget):
(WebCore::MockMediaPlayerMediaSource::seekWithTolerance): Deleted.
(WebCore::MockMediaPlayerMediaSource::waitForSeekCompleted): Deleted.
(WebCore::MockMediaPlayerMediaSource::seekCompleted): Deleted.
* Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.h:
(WebCore::MockMediaPlayerMediaSource::mediaPlayerLogIdentifier): Deleted.
(WebCore::MockMediaPlayerMediaSource::mediaPlayerLogger): Deleted.
* Source/WebCore/platform/mock/mediasource/MockMediaSourcePrivate.cpp:
(WebCore::MockMediaSourcePrivate::seekToTarget):
(WebCore::MockMediaSourcePrivate::waitForSeekCompleted): Deleted.
(WebCore::MockMediaSourcePrivate::seekCompleted): Deleted.
(WebCore::MockMediaSourcePrivate::seekToTime): Deleted.
* Source/WebCore/platform/mock/mediasource/MockMediaSourcePrivate.h:
* Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.cpp:
(WebCore::MockSourceBufferPrivate::isSeeking const): Deleted.
* Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.h:
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp:
(WebKit::RemoteMediaPlayerProxy::seekToTarget):
(WebKit::RemoteMediaPlayerProxy::updateCachedState):
(WebKit::RemoteMediaPlayerProxy::seek): Deleted.
(WebKit::RemoteMediaPlayerProxy::seekWithTolerance): Deleted.
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h:
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
* Source/WebKit/GPUProcess/media/RemoteMediaSourceProxy.cpp:
(WebKit::RemoteMediaSourceProxy::seekToTarget):
(WebKit::RemoteMediaSourceProxy::seekToTime): Deleted.
(WebKit::RemoteMediaSourceProxy::setIsSeeking): Deleted.
(WebKit::RemoteMediaSourceProxy::waitForSeekCompleted): Deleted.
(WebKit::RemoteMediaSourceProxy::seekCompleted): Deleted.
* Source/WebKit/GPUProcess/media/RemoteMediaSourceProxy.h:
* Source/WebKit/GPUProcess/media/RemoteMediaSourceProxy.messages.in:
* Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.cpp:
(WebKit::RemoteSourceBufferProxy::seekToTarget):
(WebKit::RemoteSourceBufferProxy::seekToTime): Deleted.
* Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.h:
* Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.messages.in:
* Source/WebKit/Scripts/webkit/messages.py:
(headers_for_type):
* Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in:
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
(WebKit::MediaPlayerPrivateRemote::seekToTarget):
(WebKit::MediaPlayerPrivateRemote::timeChanged):
(WebKit::MediaPlayerPrivateRemote::seeking const):
(WebKit::MediaPlayerPrivateRemote::seek): Deleted.
(WebKit::MediaPlayerPrivateRemote::seekWithTolerance): Deleted.
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
* Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.cpp:
(WebKit::MediaSourcePrivateRemote::seekToTarget):
(WebKit::MediaSourcePrivateRemote::setIsSeeking): Deleted.
(WebKit::MediaSourcePrivateRemote::waitForSeekCompleted): Deleted.
(WebKit::MediaSourcePrivateRemote::seekCompleted): Deleted.
(WebKit::MediaSourcePrivateRemote::seekToTime): Deleted.
* Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.h:
* Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.messages.in:
* Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerState.h:
* Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerState.serialization.in:
* Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.cpp:
(WebKit::SourceBufferPrivateRemote::seekToTarget):
(WebKit::SourceBufferPrivateRemote::seekToTime): Deleted.
* Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.h:

Canonical link: https://commits.webkit.org/267279@main

More information about the webkit-changes mailing list