[webkit-changes] [WebKit/WebKit] 2ee992: Provide option to "detach" a MediaSource element i...

Sat Sep 21 01:45:19 PDT 2024

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 2ee9927bd954263ac537fd27463d167c87de05e3
      https://github.com/WebKit/WebKit/commit/2ee9927bd954263ac537fd27463d167c87de05e3
  Author: Jean-Yves Avenard <jya at apple.com>
  Date:   2024-09-21 (Sat, 21 Sep 2024)

  Changed paths:
    A LayoutTests/media/media-source/media-detachablemse-append-expected.txt
    A LayoutTests/media/media-source/media-detachablemse-append.html
    M LayoutTests/platform/gtk/TestExpectations
    M Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml
    M Source/WebCore/CMakeLists.txt
    M Source/WebCore/DerivedSources-input.xcfilelist
    M Source/WebCore/DerivedSources-output.xcfilelist
    M Source/WebCore/DerivedSources.make
    M Source/WebCore/Modules/mediasource/ManagedMediaSource.cpp
    M Source/WebCore/Modules/mediasource/ManagedMediaSource.h
    M Source/WebCore/Modules/mediasource/ManagedMediaSource.idl
    M Source/WebCore/Modules/mediasource/MediaSource.cpp
    M Source/WebCore/Modules/mediasource/MediaSource.h
    M Source/WebCore/Modules/mediasource/MediaSource.idl
    M Source/WebCore/Modules/mediasource/MediaSourceHandle.cpp
    M Source/WebCore/Modules/mediasource/MediaSourceHandle.h
    A Source/WebCore/Modules/mediasource/MediaSourceInit.h
    A Source/WebCore/Modules/mediasource/MediaSourceInit.idl
    M Source/WebCore/Modules/mediasource/MediaSourceInterfaceMainThread.cpp
    M Source/WebCore/Modules/mediasource/MediaSourceInterfaceMainThread.h
    M Source/WebCore/Modules/mediasource/MediaSourceInterfaceProxy.h
    M Source/WebCore/Modules/mediasource/MediaSourceInterfaceWorker.cpp
    M Source/WebCore/Modules/mediasource/MediaSourceInterfaceWorker.h
    M Source/WebCore/Modules/mediasource/SourceBuffer.cpp
    M Source/WebCore/Modules/mediasource/SourceBuffer.h
    M Source/WebCore/Modules/mediasource/SourceBufferList.cpp
    M Source/WebCore/Modules/mediasource/SourceBufferList.h
    M Source/WebCore/Sources.txt
    M Source/WebCore/WebCore.xcodeproj/project.pbxproj
    M Source/WebCore/html/HTMLMediaElement.cpp
    M Source/WebCore/platform/graphics/MediaPlayer.cpp
    M Source/WebCore/platform/graphics/MediaPlayer.h
    M Source/WebCore/platform/graphics/MediaPlayerPrivate.h
    M Source/WebCore/platform/graphics/MediaSourcePrivate.cpp
    M Source/WebCore/platform/graphics/MediaSourcePrivate.h
    M Source/WebCore/platform/graphics/MediaSourcePrivateClient.h
    M Source/WebCore/platform/graphics/SourceBufferPrivate.cpp
    M Source/WebCore/platform/graphics/SourceBufferPrivate.h
    M Source/WebCore/platform/graphics/SourceBufferPrivateClient.h
    M Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.mm
    M Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm
    M Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.h
    M Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h
    M Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp
    M Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h
    M Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.cpp
    M Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.h
    M Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.cpp
    M Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.h
    M Source/WebCore/platform/graphics/holepunch/MediaPlayerPrivateHolePunch.h
    M Source/WebCore/platform/graphics/win/MediaPlayerPrivateMediaFoundation.h
    M Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.cpp
    M Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.h
    M Source/WebCore/platform/mock/mediasource/MockMediaSourcePrivate.cpp
    M Source/WebCore/platform/mock/mediasource/MockMediaSourcePrivate.h
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerManagerProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerManagerProxy.h
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteMediaSourceProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteMediaSourceProxy.h
    M Source/WebKit/GPUProcess/media/RemoteMediaSourceProxy.messages.in
    M Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.h
    M Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.messages.in
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h
    M Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.h
    M Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemoteMessageReceiver.messages.in
    M Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.h
    M Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemoteMessageReceiver.messages.in

  Log Message:
  -----------
  Provide option to "detach" a MediaSource element in a non-destructive fashion.
https://bugs.webkit.org/show_bug.cgi?id=279875
rdar://129298010

Reviewed by Jer Noble.

A MediaSource object is attached to a media element. When this MediaSource is detached from the media element (such as when the src or srcObject attribute is cleared or set to another object), the “3.15.2 Detaching from a media element” algorithm is to be run which will clear all its source buffers and their content and change the readyState of the MediaSource object to “close”

Discussion with users has show interests on having the ability to detach a MediaSource from a media element temporarily to re-attach it again later so that a new MediaSource doesn’t need to be created and having to reload all the original content again.

We add a construction parameter allowing to construct a MediaSource or ManagedMediaSource that is detachable.

If detached from the media element, a detachable MediaSource's content will not be destructed and can later be re-attached to the same media element (or another).
This option is disabled by default and change isn't observable.
Added test and covered by other tests.

* LayoutTests/media/media-source/media-detachablemse-append-expected.txt: Added.
* LayoutTests/media/media-source/media-detachablemse-append.html: Added.
* Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml:
* Source/WebCore/CMakeLists.txt:
* Source/WebCore/DerivedSources-input.xcfilelist:
* Source/WebCore/DerivedSources-output.xcfilelist:
* Source/WebCore/DerivedSources.make:
* Source/WebCore/Modules/mediasource/ManagedMediaSource.cpp:
(WebCore::ManagedMediaSource::create):
(WebCore::ManagedMediaSource::ManagedMediaSource):
* Source/WebCore/Modules/mediasource/ManagedMediaSource.h:
* Source/WebCore/Modules/mediasource/ManagedMediaSource.idl:
* Source/WebCore/Modules/mediasource/MediaSource.cpp:
(WebCore::MediaSource::create):
(WebCore::MediaSource::MediaSource):
(WebCore::MediaSource::~MediaSource):
(WebCore::MediaSource::setPrivate):
(WebCore::MediaSource::setPrivateAndOpen):
(WebCore::MediaSource::reOpen):
(WebCore::MediaSource::open):
(WebCore::MediaSource::duration const):
(WebCore::MediaSource::setReadyState):
(WebCore::MediaSource::detachFromElement):
(WebCore::MediaSource::openIfDeferredOpen):
(WebCore::MediaSource::stop):
(WebCore::MediaSource::onReadyStateChange):
(WebCore::MediaSource::regenerateActiveSourceBuffers):
(WebCore::MediaSource::handle):
* Source/WebCore/Modules/mediasource/MediaSource.h:
(WebCore::MediaSource::detachable const):
* Source/WebCore/Modules/mediasource/MediaSource.idl:
* Source/WebCore/Modules/mediasource/MediaSourceHandle.cpp:
(WebCore::MediaSourceHandle::create):
(WebCore::MediaSourceHandle::MediaSourceHandle):
* Source/WebCore/Modules/mediasource/MediaSourceHandle.h:
(WebCore::MediaSourceHandle::detachable const):
* Source/WebCore/Modules/mediasource/MediaSourceInit.h: Copied from Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemoteMessageReceiver.messages.in.
* Source/WebCore/Modules/mediasource/MediaSourceInit.idl: Copied from Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemoteMessageReceiver.messages.in.
* Source/WebCore/Modules/mediasource/MediaSourceInterfaceMainThread.cpp:
(WebCore::MediaSourceInterfaceMainThread::detachable const):
* Source/WebCore/Modules/mediasource/MediaSourceInterfaceMainThread.h:
* Source/WebCore/Modules/mediasource/MediaSourceInterfaceProxy.h:
* Source/WebCore/Modules/mediasource/MediaSourceInterfaceWorker.cpp:
(WebCore::MediaSourceInterfaceWorker::detachable const):
* Source/WebCore/Modules/mediasource/MediaSourceInterfaceWorker.h:
* Source/WebCore/Modules/mediasource/SourceBuffer.cpp:
(WebCore::SourceBuffer::detached):
(WebCore::SourceBuffer::attached):
(WebCore::SourceBuffer::sourceBufferPrivateDidAttach):
* Source/WebCore/Modules/mediasource/SourceBuffer.h:
* Source/WebCore/Modules/mediasource/SourceBufferList.cpp:
(WebCore::SourceBufferList::replaceWith):
(WebCore::SourceBufferList::swap): Deleted.
* Source/WebCore/Modules/mediasource/SourceBufferList.h:
* Source/WebCore/Sources.txt:
* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::loadResource):
* Source/WebCore/platform/graphics/MediaPlayer.cpp:
* Source/WebCore/platform/graphics/MediaPlayer.h:
* Source/WebCore/platform/graphics/MediaPlayerPrivate.h:
* Source/WebCore/platform/graphics/MediaSourcePrivate.cpp:
(WebCore::MediaSourcePrivate::shutdown):
* Source/WebCore/platform/graphics/MediaSourcePrivate.h:
* Source/WebCore/platform/graphics/MediaSourcePrivateClient.h:
* Source/WebCore/platform/graphics/SourceBufferPrivate.cpp:
(WebCore::SourceBufferPrivate::didReceiveInitializationSegment):
(WebCore::SourceBufferPrivate::attached):
* Source/WebCore/platform/graphics/SourceBufferPrivate.h:
* Source/WebCore/platform/graphics/SourceBufferPrivateClient.h:
* Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h:
(isType):
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::load):
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.mm:
(WebCore::MediaSourcePrivateAVFObjC::setPlayer):
* Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
(WebCore::SourceBufferPrivateAVFObjC::player const):
* Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.h:
* Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:
(isType):
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp:
(WebCore::MediaPlayerPrivateGStreamerMSE::load):
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h:
(isType):
* Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.cpp:
(WebCore::MediaSourcePrivateGStreamer::addSourceBuffer):
(WebCore::MediaSourcePrivateGStreamer::player const):
(WebCore::MediaSourcePrivateGStreamer::setPlayer):
(WebCore::MediaSourcePrivateGStreamer::durationChanged):
(WebCore::MediaSourcePrivateGStreamer::markEndOfStream):
(WebCore::MediaSourcePrivateGStreamer::unmarkEndOfStream):
(WebCore::MediaSourcePrivateGStreamer::mediaPlayerReadyState const):
(WebCore::MediaSourcePrivateGStreamer::setMediaPlayerReadyState):
(WebCore::MediaSourcePrivateGStreamer::startPlaybackIfHasAllTracks):
(WebCore::MediaSourcePrivateGStreamer::notifyActiveSourceBuffersChanged):
* Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.h:  Don't store a reference to the MediaPlayerPrivate as it will be deleted.
Instead we use ThreadSafeWeakPtr and provide a fallible method to check if the player is still present.
* Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.cpp:
(WebCore::SourceBufferPrivateGStreamer::create):
(WebCore::SourceBufferPrivateGStreamer::SourceBufferPrivateGStreamer):
(WebCore::SourceBufferPrivateGStreamer::appendInternal):
(WebCore::SourceBufferPrivateGStreamer::resetParserStateInternal):
(WebCore::SourceBufferPrivateGStreamer::flush):
(WebCore::SourceBufferPrivateGStreamer::enqueueSample):
(WebCore::SourceBufferPrivateGStreamer::isReadyForMoreSamples):
(WebCore::SourceBufferPrivateGStreamer::allSamplesInTrackEnqueued):
(WebCore::SourceBufferPrivateGStreamer::player const):
* Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.h:  No longer store a reference to the MediaPlayerPrivate as
it will be deleted when the mediasource is detached from the media element.
* Source/WebCore/platform/graphics/holepunch/MediaPlayerPrivateHolePunch.h:
* Source/WebCore/platform/graphics/win/MediaPlayerPrivateMediaFoundation.h:
* Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.cpp:
(WebCore::MockMediaPlayerMediaSource::load):
* Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.h:
(isType):
* Source/WebCore/platform/mock/mediasource/MockMediaSourcePrivate.cpp:
(WebCore::MockMediaSourcePrivate::setPlayer):
* Source/WebCore/platform/mock/mediasource/MockMediaSourcePrivate.h:
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerManagerProxy.cpp:
(WebKit::RemoteMediaPlayerManagerProxy::clear):
(WebKit::RemoteMediaPlayerManagerProxy::registerMediaSource):
(WebKit::RemoteMediaPlayerManagerProxy::invalidateMediaSource):
(WebKit::RemoteMediaPlayerManagerProxy::pendingMediaSource):
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerManagerProxy.h:
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp:
(WebKit::RemoteMediaPlayerProxy::~RemoteMediaPlayerProxy):
(WebKit::RemoteMediaPlayerProxy::loadMediaSource):
* Source/WebKit/GPUProcess/media/RemoteMediaSourceProxy.cpp:
(WebKit::RemoteMediaSourceProxy::RemoteMediaSourceProxy):
(WebKit::RemoteMediaSourceProxy::setMediaPlayers):
(WebKit::RemoteMediaSourceProxy::disconnect):
(WebKit::RemoteMediaSourceProxy::reOpen):
(WebKit::RemoteMediaSourceProxy::waitForTarget):
(WebKit::RemoteMediaSourceProxy::seekToTime):
(WebKit::RemoteMediaSourceProxy::addSourceBuffer):
(WebKit::RemoteMediaSourceProxy::attached):
(WebKit::RemoteMediaSourceProxy::shutdown):
(WebKit::RemoteMediaSourceProxy::connectionToWebProcess const):
* Source/WebKit/GPUProcess/media/RemoteMediaSourceProxy.h:
* Source/WebKit/GPUProcess/media/RemoteMediaSourceProxy.messages.in:
* Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.cpp:
(WebKit::RemoteSourceBufferProxy::setMediaPlayer):
(WebKit::RemoteSourceBufferProxy::sourceBufferPrivateDidReceiveInitializationSegment):
(WebKit::RemoteSourceBufferProxy::attached):
(WebKit::RemoteSourceBufferProxy::sourceBufferPrivateDidAttach):
(WebKit::RemoteSourceBufferProxy::createInitializationSegmentInfo):
(WebKit::RemoteSourceBufferProxy::shutdown): Deleted.
* Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.h:
* Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.messages.in:
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
(WebKit::MediaPlayerPrivateRemote::load):
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
(isType):
* Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.cpp:
(WebKit::MediaSourcePrivateRemote::addSourceBuffer):
(WebKit::MediaSourcePrivateRemote::setPlayer):
(WebKit::MediaSourcePrivateRemote::shutdown):
(WebKit::MediaSourcePrivateRemote::MessageReceiver::mediaSourcePrivateShuttingDown): Deleted.
* Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemote.h:
(isType):
* Source/WebKit/WebProcess/GPU/media/MediaSourcePrivateRemoteMessageReceiver.messages.in:
* Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.cpp:
(WebKit::SourceBufferPrivateRemote::create):
(WebKit::SourceBufferPrivateRemote::SourceBufferPrivateRemote):
(WebKit::SourceBufferPrivateRemote::MessageReceiver::sourceBufferPrivateDidReceiveInitializationSegment):
(WebKit::SourceBufferPrivateRemote::MessageReceiver::sourceBufferPrivateDidAttach):
(WebKit::SourceBufferPrivateRemote::MessageReceiver::createInitializationSegment const):
(WebKit::SourceBufferPrivateRemote::totalTrackBufferSizeInBytes const):
(WebKit::SourceBufferPrivateRemote::player const):
(WebKit::SourceBufferPrivateRemote::attached):
(WebKit::SourceBufferPrivateRemote::MessageReceiver::sourceBufferPrivateShuttingDown): Deleted.
* Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.h:
* Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemoteMessageReceiver.messages.in:

Canonical link: https://commits.webkit.org/284032@main

To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications