[webkit-changes] [WebKit/WebKit] eaa411: [MSE] screen should sleep if the video has stalled...

[Jean-Yves Avenard]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: eaa411f285640e50b506f871981eed87e6e91fa3
      https://github.com/WebKit/WebKit/commit/eaa411f285640e50b506f871981eed87e6e91fa3
  Author: Jean-Yves Avenard <jya at apple.com>
  Date:   2024-03-08 (Fri, 08 Mar 2024)

  Changed paths:
    M LayoutTests/platform/mac-wk2/TestExpectations
    M Source/WebCore/html/HTMLMediaElement.cpp
    M Source/WebCore/platform/graphics/MediaPlayer.cpp
    M Source/WebCore/platform/graphics/MediaPlayer.h
    M Source/WebCore/platform/graphics/MediaPlayerPrivate.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm
    M Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.h
    M Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.mm
    M Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp
    M Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h
    M Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.cpp
    M Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.h
    M Source/WebKit/CMakeLists.txt
    M Source/WebKit/DerivedSources-input.xcfilelist
    M Source/WebKit/DerivedSources.make
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in
    M Source/WebKit/Scripts/webkit/messages.py
    M Source/WebKit/WebKit.xcodeproj/project.pbxproj
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.messages.in
    A Source/WebKit/WebProcess/GPU/media/MediaTimeUpdateData.serialization.in

  Log Message:
  -----------
  [MSE] screen should sleep if the video has stalled, not just paused
https://bugs.webkit.org/show_bug.cgi?id=270620
rdar://124190917

Reviewed by Youenn Fablet.

A video may be playing as far as the HTMLMediaElement is concerned, but has stalled.
Those videos should allow the screen to sleep.

The code relied on an undocumented requirement that MediaPlayer::paused() returned
true if the video has stalled, but this requirement causes breakage elsewhere (see
webkit.org/b/270614)

Checking on the value of MediaPlayerPrivateRemote::timeIsProgressing instead
of paused() made several tests fail. Each failure revealed a discrepency
between the GPU's MediaPlayer's time and the simulated currentTime in the WP.
To avoid those, we re-sync the time estimator in each messages sent by the
GPU process (currentTime, wall time, and if media player's time is progressing).

No change in observable behaviour at this stage.
Covered by existing tests.

* LayoutTests/platform/mac-wk2/TestExpectations:
* Source/WebCore/html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::shouldDisableSleep const):
* Source/WebCore/platform/graphics/MediaPlayer.cpp:
(WebCore::MediaPlayer::timeIsProgressing const):
(WebCore::MediaPlayer::currentTimeMayProgress const): Deleted.
* Source/WebCore/platform/graphics/MediaPlayer.h:
* Source/WebCore/platform/graphics/MediaPlayerPrivate.h:
(WebCore::MediaPlayerPrivateInterface::timeIsProgressing const):
(WebCore::MediaPlayerPrivateInterface::currentTimeMayProgress const): Deleted.
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::timeIsProgressing const):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::currentTimeMayProgress const): Deleted.
* Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.h:
* Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.mm:
(WebCore::MediaPlayerPrivateWebM::timeIsProgressing const):
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp:
(WebCore::MediaPlayerPrivateGStreamerMSE::timeIsProgressing const):
(WebCore::MediaPlayerPrivateGStreamerMSE::currentTimeMayProgress const): Deleted.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h:
* Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.cpp:
(WebCore::MockMediaPlayerMediaSource::timeIsProgressing const):
(WebCore::MockMediaPlayerMediaSource::currentTimeMayProgress const): Deleted.
* Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.h:
* Source/WebKit/DerivedSources-input.xcfilelist:
* Source/WebKit/DerivedSources.make:
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp:
(WebKit::RemoteMediaPlayerProxy::~RemoteMediaPlayerProxy):
(WebKit::timeUpdateData const):
(WebKit::RemoteMediaPlayerProxy::mediaPlayerSeeked):
(WebKit::RemoteMediaPlayerProxy::mediaPlayerTimeChanged):
(WebKit::RemoteMediaPlayerProxy::mediaPlayerRateChanged):
(WebKit::RemoteMediaPlayerProxy::mediaPlayerPlaybackStateChanged):
(WebKit::RemoteMediaPlayerProxy::currentTimeChanged):
(WebKit::RemoteMediaPlayerProxy::performTaskAtTime): Code was taking into account potential delay
between the time the WP sent a message and the GPUP received it. However
the time given to run the task is in the GPUP's time referential already
and as such, shouldn't be adjusted. Additionally, if the time sent had
already been in the past, it would have caused the simulated time to also
go backward which was incorrect.
(WebKit::RemoteMediaPlayerProxy::mediaPlayerPausedOrStalled const): Deleted.
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h:
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
* Source/WebKit/Scripts/webkit/messages.py:
(headers_for_type):
* Source/WebKit/WebKit.xcodeproj/project.pbxproj:
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
(WebKit::MediaPlayerPrivateRemote::TimeProgressEstimator::setTime):
(WebKit::MediaPlayerPrivateRemote::timeIsProgressing const):
(WebKit::MediaPlayerPrivateRemote::seekToTarget):
(WebKit::MediaPlayerPrivateRemote::seeked):
(WebKit::MediaPlayerPrivateRemote::timeChanged):
(WebKit::MediaPlayerPrivateRemote::rateChanged):
(WebKit::MediaPlayerPrivateRemote::playbackStateChanged):
(WebKit::MediaPlayerPrivateRemote::currentTimeChanged):
(WebKit::MediaPlayerPrivateRemote::performTaskAtTime):
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.messages.in:
* Source/WebKit/WebProcess/GPU/media/MediaTimeUpdateData.serialization.in: Added.

Canonical link: https://commits.webkit.org/275831@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications