[webkit-changes] [WebKit/WebKit] f820f4: [GPUP][MSE] incorrect handling when multiple init ...

[Jean-Yves Avenard]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: f820f4ae85677af2baa1fc8f50d354c3bfc56fc0
      https://github.com/WebKit/WebKit/commit/f820f4ae85677af2baa1fc8f50d354c3bfc56fc0
  Author: Jean-Yves Avenard <jya at apple.com>
  Date:   2023-03-29 (Wed, 29 Mar 2023)

  Changed paths:
    A LayoutTests/media/media-source/media-source-multiple-concurrent-initialization-segments-expected.txt
    A LayoutTests/media/media-source/media-source-multiple-concurrent-initialization-segments.html
    M LayoutTests/media/media-source/media-source-stpp-crash-expected.txt
    M LayoutTests/media/media-source/media-source-stpp-crash.html
    M Source/WebCore/Modules/mediasource/SourceBuffer.cpp
    M Source/WebCore/Modules/mediasource/SourceBuffer.h
    M Source/WebCore/platform/graphics/AudioTrackPrivate.h
    M Source/WebCore/platform/graphics/InbandTextTrackPrivate.h
    M Source/WebCore/platform/graphics/SourceBufferPrivate.cpp
    M Source/WebCore/platform/graphics/SourceBufferPrivate.h
    M Source/WebCore/platform/graphics/SourceBufferPrivateClient.h
    M Source/WebCore/platform/graphics/TrackPrivateBase.h
    M Source/WebCore/platform/graphics/VideoTrackPrivate.h
    M Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm
    M Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.cpp
    M Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.h
    M Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.cpp
    M Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.h
    M Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.h
    M Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.messages.in

  Log Message:
  -----------
  [GPUP][MSE] incorrect handling when multiple init segments are processed during a single appendBuffer
https://bugs.webkit.org/show_bug.cgi?id=254079
rdar://106859839

Reviewed by Youenn Fablet.

It is necessary to tie the Media Segment being parsed to the right Init Segment.
To do so, we queue all samples and init segments found in the input buffer
until they have all been parsed by the SourceBufferPrivate.
We processed then the next init segment if any, initialize it and await
for the initialization to complete.
Once done, we will process all Media Segment tie to this Init Segment.
Once done, we repeat these steps with the next Init Segment in the Input
Buffer if any.

Tests:
* LayoutTests/media/media-source/media-source-multiple-concurrent-initialization-segments-expected.txt: Added.
* LayoutTests/media/media-source/media-source-multiple-concurrent-initialization-segments.html: Added.

* LayoutTests/media/media-source/media-source-stpp-crash-expected.txt:
* LayoutTests/media/media-source/media-source-stpp-crash.html: Amend test to explicitly show that error event should be fired.
* Source/WebCore/Modules/mediasource/SourceBuffer.cpp:
(WebCore::m_logIdentifier):
(WebCore::SourceBuffer::~SourceBuffer):
(WebCore::SourceBuffer::removedFromMediaSource):
(WebCore::SourceBuffer::sourceBufferPrivateDidReceiveInitializationSegment): Streamline handling of errors,
having it all managed by sourceBufferPrivateAppendComplete. It also prevents the MediaSource to be deleted and
the SourceBuffer be detached from the SourceBufferPrivate while the method is running should there be an error.
(WebCore::SourceBuffer::sourceBufferPrivateAppendError): Deleted.
* Source/WebCore/Modules/mediasource/SourceBuffer.h:
* Source/WebCore/platform/graphics/AudioTrackPrivate.h:
(isType):
* Source/WebCore/platform/graphics/InbandTextTrackPrivate.h:
(isType):
* Source/WebCore/platform/graphics/SourceBufferPrivate.cpp:
(WebCore::SourceBufferPrivate::~SourceBufferPrivate):
(WebCore::SourceBufferPrivate::updateHighestPresentationTimestamp):
(WebCore::SourceBufferPrivate::setBufferedRanges):
(WebCore::SourceBufferPrivate::updateBufferedFromTrackBuffers):
(WebCore::SourceBufferPrivate::appendCompleted):
(WebCore::SourceBufferPrivate::reenqueSamples):
(WebCore::SourceBufferPrivate::clearTrackBuffers):
(WebCore::SourceBufferPrivate::fastSeekTimeForMediaTime):
(WebCore::SourceBufferPrivate::provideMediaData):
(WebCore::SourceBufferPrivate::removeCodedFrames):
(WebCore::SourceBufferPrivate::evictCodedFrames):
(WebCore::SourceBufferPrivate::setClient): Simplify detection that we've been detached
(WebCore::SourceBufferPrivate::detach): removing redundant member variable.
(WebCore::SourceBufferPrivate::isAttached const): Rely on m_client value to check if we're still attached to SourceBuffer.
(WebCore::SourceBufferPrivate::didReceiveInitializationSegment):
(WebCore::SourceBufferPrivate::didReceiveSample):
(WebCore::SourceBufferPrivate::append):
(WebCore::SourceBufferPrivate::processPendingOperations):
(WebCore::SourceBufferPrivate::abortPendingOperations):
(WebCore::SourceBufferPrivate::processInitSegment):
(WebCore::SourceBufferPrivate::processMediaSample):
(WebCore::SourceBufferPrivate::processMediaSamples):
(WebCore::SourceBufferPrivate::resetParserState):
(WebCore::SourceBufferPrivate::didReceiveSampleForTrackId): Deleted.
(WebCore::SourceBufferPrivate::processPendingSamples): Deleted.
* Source/WebCore/platform/graphics/SourceBufferPrivate.h:
(WebCore::SourceBufferPrivate::isMediaSampleAllowed const):
(WebCore::SourceBufferPrivate::setIsAttached): Deleted.
(WebCore::SourceBufferPrivate::processingInitializationSegment const): Deleted.
* Source/WebCore/platform/graphics/SourceBufferPrivateClient.h:
(WebCore::SourceBufferPrivateClient::isAsync const):
* Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.h:
* Source/WebCore/platform/graphics/TrackPrivateBase.h: Allow dynamic casting for Tracks objects.
* Source/WebCore/platform/graphics/VideoTrackPrivate.h:
(isType):
* Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h:
* Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
(WebCore::SourceBufferPrivateAVFObjC::setTrackChangeCallbacks):
(WebCore::SourceBufferPrivateAVFObjC::didParseInitializationData):
(WebCore::SourceBufferPrivateAVFObjC::didProvideMediaDataForTrackId):
(WebCore::SourceBufferPrivateAVFObjC::isMediaSampleAllowed const):
(WebCore::SourceBufferPrivateAVFObjC::didProvideContentKeyRequestInitializationDataForTrackID):
(WebCore::SourceBufferPrivateAVFObjC::enqueueSample):
(WebCore::SourceBufferPrivateAVFObjC::processPendingTrackChangeTasks): Deleted.
(WebCore::SourceBufferPrivateAVFObjC::didReceiveSampleForTrackId): Deleted.
* Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.cpp:
(WebCore::SourceBufferPrivateGStreamer::didReceiveInitializationSegment):
* Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.h:
* Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.cpp:
(WebCore::MockSourceBufferPrivate::didReceiveInitializationSegment):
* Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.cpp:
(WebKit::RemoteSourceBufferProxy::RemoteSourceBufferProxy):
(WebKit::RemoteSourceBufferProxy::~RemoteSourceBufferProxy):
(WebKit::RemoteSourceBufferProxy::sourceBufferPrivateAppendError): Deleted.
* Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.h:
* Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.cpp:
(WebKit::SourceBufferPrivateRemote::sourceBufferPrivateAppendError): Deleted.
* Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.h:
* Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.messages.in: Remove unnecessary method.

Canonical link: https://commits.webkit.org/262300@main