[webkit-changes] [WebKit/WebKit] b0908a: [MSE] Remove HTMLMedia Element access from MediaSo...

Sun Feb 11 03:04:06 PST 2024

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: b0908a7024e145b4957d96953f1576ce62db91a8
      https://github.com/WebKit/WebKit/commit/b0908a7024e145b4957d96953f1576ce62db91a8
  Author: Jean-Yves Avenard <jya at apple.com>
  Date:   2024-02-11 (Sun, 11 Feb 2024)

  Changed paths:
    M Source/WebCore/Modules/mediasource/ManagedMediaSource.cpp
    M Source/WebCore/Modules/mediasource/ManagedMediaSource.h
    M Source/WebCore/Modules/mediasource/MediaSource.cpp
    M Source/WebCore/Modules/mediasource/MediaSource.h
    M Source/WebCore/Modules/mediasource/SourceBuffer.cpp
    M Source/WebCore/Modules/mediasource/SourceBuffer.h
    M Source/WebCore/html/HTMLMediaElement.cpp
    M Source/WebCore/html/HTMLMediaElement.h
    M Source/WebCore/html/MediaElementSession.cpp
    M Source/WebCore/html/MediaElementSession.h
    M Source/WebCore/platform/graphics/MediaPlayer.cpp
    M Source/WebCore/platform/graphics/MediaPlayer.h
    M Source/WebCore/platform/graphics/MediaPlayerPrivate.cpp
    M Source/WebCore/platform/graphics/MediaPlayerPrivate.h
    M Source/WebCore/platform/graphics/MediaSourcePrivate.cpp
    M Source/WebCore/platform/graphics/MediaSourcePrivate.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.mm
    M Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm
    M Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h
    M Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.cpp
    M Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.h
    M Source/WebCore/platform/graphics/win/MediaPlayerPrivateMediaFoundation.cpp
    M Source/WebCore/platform/graphics/win/MediaPlayerPrivateMediaFoundation.h
    M Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.cpp
    M Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.h
    M Source/WebCore/platform/mock/mediasource/MockMediaSourcePrivate.cpp
    M Source/WebCore/platform/mock/mediasource/MockMediaSourcePrivate.h
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h
    M Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.h

  Log Message:
  -----------
  [MSE] Remove HTMLMedia Element access from MediaSource
https://bugs.webkit.org/show_bug.cgi?id=268796
rdar://122359957

Reviewed by Eric Carlson.

If the MediaSource object lives in a worker thread, it will not be possible
for it to access the HTMLMediaElement directly.
We remove most access from the MediaSource to the media element.

One attribute accessed often is the HTMLMediaElement's currentTime value.
Instead we access the MediaPlayerPrivate value. As the HTMLMediaElement returns
the seekTime if a seek has been started we need a way to reproduce a similar
behaviour from the MediaPlayerPrivate. We add a `willSeekToTarget()` and `pendingSeekTime()`
method that will receive the seek time from the HTMLMediaElement.
To differentiate the behaviour between the actual player's time and what the
time according to the media element is, we add a `currentOrPendingSeekTime` method.

The handling of the Tracks hasn't been converted yet and will be done in
a follow-up change.

The MediaSourcePrivate::mediaTime method was duplicated in all implementations.
To simplify, we move it to the base class.

Fly-By: there were 3 virtual methods in the MediaPlayerPrivateInterface class
to retrieve the current time. We only leave on: currentMediaTime.
Fly-By: the static implementation to retrieve a SourceBuffer maximum size was
living in the MediaElementSession. We move it to the SourceBuffer instead.
Fly-By: The MockMediaPlayerMediaSource::isSeeking() would have always returned true
before seeking at least once. Correct logic.

Covered by existing tests, no change in observable behaviour.

* Source/WebCore/Modules/mediasource/ManagedMediaSource.cpp:
(WebCore::ManagedMediaSource::isOpen const): Deleted.
(WebCore::ManagedMediaSource::ensurePrefsRead):
(WebCore::ManagedMediaSource::monitorSourceBuffers):
(WebCore::ManagedMediaSource::isOpen const):
* Source/WebCore/Modules/mediasource/MediaSource.cpp:
(WebCore::MediaSource::setPrivateAndOpen):
(WebCore::MediaSource::currentMediaTime const):
For consistency with other classes naming, rename currentTime() to currentMediaTime().
(WebCore::MediaSource::buffered const):
(WebCore::MediaSource::completeSeek):
(WebCore::MediaSource::hasBufferedTime):
(WebCore::MediaSource::hasCurrentTime):
(WebCore::MediaSource::hasFutureTime):
(WebCore::MediaSource::streamEndedWithError):
(WebCore::MediaSource::addSourceBuffer):
(WebCore::MediaSource::removeSourceBuffer):
(WebCore::MediaSource::detachFromElement):
(WebCore::MediaSource::attachToElement):
(WebCore::MediaSource::settings const):
(WebCore::MediaSource::stop):
(WebCore::MediaSource::notifyElementUpdateMediaState const):
(WebCore::MediaSource::incrementDroppedFrameCount):
(WebCore::MediaSource::addAudioTrackToElement):
(WebCore::MediaSource::addTextTrackToElement):
(WebCore::MediaSource::addVideoTrackToElement):
(WebCore::MediaSource::currentTime const): Deleted.
* Source/WebCore/Modules/mediasource/MediaSource.h:
(WebCore::MediaSource::mediaElement const): Deleted.
* Source/WebCore/Modules/mediasource/SourceBuffer.cpp:
(WebCore::SourceBuffer::rangeRemoval):
(WebCore::SourceBuffer::changeType):
(WebCore::SourceBuffer::appendBufferInternal):
(WebCore::SourceBuffer::sourceBufferPrivateAppendComplete):
(WebCore::SourceBuffer::maximumBufferSize const):
(WebCore::SourceBuffer::sourceBufferPrivateDidReceiveInitializationSegment):
(WebCore::SourceBuffer::sourceBufferPrivateDidDropSample):
(WebCore::SourceBuffer::canPlayThroughRange):
(WebCore::SourceBuffer::settings const):
(WebCore::SourceBuffer::memoryPressure):
* Source/WebCore/Modules/mediasource/SourceBuffer.h:
* Source/WebCore/html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::deferredMediaSourceOpenCanProgress const):
move check to see if the ManagedMediaSource can progress to open to the HTMLMediaElement class.
(WebCore::HTMLMediaElement::seekWithTolerance):
(WebCore::HTMLMediaElement::seekTask):
(WebCore::HTMLMediaElement::clearSeeking):
(WebCore::HTMLMediaElement::maximumSourceBufferSize const): Deleted.
* Source/WebCore/html/HTMLMediaElement.h:
(WebCore::HTMLMediaElement::incrementDroppedFrameCount):
* Source/WebCore/html/MediaElementSession.cpp:
(WebCore::MediaElementSession::maximumMediaSourceBufferSize const): Deleted.
* Source/WebCore/html/MediaElementSession.h:
* Source/WebCore/platform/graphics/MediaPlayer.cpp:
(WebCore::MediaPlayer::willSeekToTarget):
* Source/WebCore/platform/graphics/MediaPlayer.h:
* Source/WebCore/platform/graphics/MediaPlayerPrivate.cpp:
(WebCore::MediaPlayerPrivateInterface::currentOrPendingSeekTime const):
* Source/WebCore/platform/graphics/MediaPlayerPrivate.h:
(WebCore::MediaPlayerPrivateInterface::currentMediaTime const):
(WebCore::MediaPlayerPrivateInterface::willSeekToTarget):
(WebCore::MediaPlayerPrivateInterface::pendingSeekTime const):
(WebCore::MediaPlayerPrivateInterface::currentTime const): Deleted.
(WebCore::MediaPlayerPrivateInterface::currentTimeDouble const): Deleted.
* Source/WebCore/platform/graphics/MediaSourcePrivate.cpp:
(WebCore::MediaSourcePrivate::currentMediaTime const):
* Source/WebCore/platform/graphics/MediaSourcePrivate.h:
To allow for a common currentMediaTime() implementation we need a generic
way to access the associated MediaPlayerPrivate.
Add a pure virtual `parent()`.
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
(WebCore::MediaPlayerPrivateAVFoundationObjC::currentTime const): Deleted.
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::currentMediaTime const):
While seeking, return the last seekTime.
* Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.mm:
(WebCore::MediaSourcePrivateAVFObjC::notifyActiveSourceBuffersChanged):
(WebCore::MediaSourcePrivateAVFObjC::player const):
(WebCore::MediaSourcePrivateAVFObjC::durationChanged):
(WebCore::MediaSourcePrivateAVFObjC::markEndOfStream):
(WebCore::MediaSourcePrivateAVFObjC::mediaPlayerReadyState const):
(WebCore::MediaSourcePrivateAVFObjC::setMediaPlayerReadyState):
(WebCore::MediaSourcePrivateAVFObjC::sourceBufferKeyNeeded):
(WebCore::MediaSourcePrivateAVFObjC::setSourceBufferWithSelectedVideo):
(WebCore::MediaSourcePrivateAVFObjC::currentMediaTime const): Deleted.
* Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
(WebCore::SourceBufferPrivateAVFObjC::player const):
* Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.cpp:
(WebCore::MediaSourcePrivateGStreamer::currentMediaTime const): Deleted.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.h:
* Source/WebCore/platform/graphics/win/MediaPlayerPrivateMediaFoundation.cpp:
Use MediaTime instead of floats.
(WebCore::MediaPlayerPrivateMediaFoundation::currentMediaTime const):
(WebCore::MediaPlayerPrivateMediaFoundation::currentTime const): Deleted.
* Source/WebCore/platform/graphics/win/MediaPlayerPrivateMediaFoundation.h:
* Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.cpp:
(WebCore::MockMediaPlayerMediaSource::seeking const): Fix logic, m_seekCompleted
was initialised to false, making seeking() return the wrong value.
(WebCore::MockMediaPlayerMediaSource::currentMediaTime const): Return last seek time
while seek hasn't completed.
(WebCore::MockMediaPlayerMediaSource::seekToTarget):
* Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.h:
* Source/WebCore/platform/mock/mediasource/MockMediaSourcePrivate.cpp:
(WebCore::MockMediaSourcePrivate::player const):
(WebCore::MockMediaSourcePrivate::currentMediaTime const): Deleted.
* Source/WebCore/platform/mock/mediasource/MockMediaSourcePrivate.h:
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
(WebKit::MediaPlayerPrivateRemote::willSeekToTarget):
(WebKit::MediaPlayerPrivateRemote::pendingSeekTime const):
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
* Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.cpp:
(WebKit::MediaSourcePrivateRemote::player const):
* Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.h:

Canonical link: https://commits.webkit.org/274442@main