[webkit-changes] [WebKit/WebKit] de9e3d: [MSE] `ASSERT(canSafelyBeUsed()); ` ASSERTION when ...

[Jean-Yves Avenard]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: de9e3d6bc4880dab143d3f4e3643adf588078dd9
      https://github.com/WebKit/WebKit/commit/de9e3d6bc4880dab143d3f4e3643adf588078dd9
  Author: Jean-Yves Avenard <jya at apple.com>
  Date:   2024-02-18 (Sun, 18 Feb 2024)

  Changed paths:
    M Source/WebCore/html/track/AudioTrack.cpp
    M Source/WebCore/html/track/InbandTextTrack.cpp
    M Source/WebCore/html/track/TrackBase.cpp
    M Source/WebCore/html/track/TrackBase.h
    M Source/WebCore/html/track/VideoTrack.cpp
    M Source/WebCore/platform/graphics/AudioTrackPrivate.h
    M Source/WebCore/platform/graphics/AudioTrackPrivateClient.h
    M Source/WebCore/platform/graphics/InbandTextTrackPrivate.h
    M Source/WebCore/platform/graphics/InbandTextTrackPrivateClient.h
    M Source/WebCore/platform/graphics/TrackPrivateBase.cpp
    M Source/WebCore/platform/graphics/TrackPrivateBase.h
    M Source/WebCore/platform/graphics/TrackPrivateBaseClient.h
    M Source/WebCore/platform/graphics/VideoTrackPrivate.h
    M Source/WebCore/platform/graphics/VideoTrackPrivateClient.h
    M Source/WebCore/platform/graphics/avfoundation/InbandMetadataTextTrackPrivateAVF.cpp
    M Source/WebCore/platform/graphics/avfoundation/InbandTextTrackPrivateAVF.cpp
    M Source/WebCore/platform/graphics/avfoundation/objc/InbandChapterTrackPrivateAVFObjC.mm
    M Source/WebCore/platform/graphics/gstreamer/AudioTrackPrivateGStreamer.cpp
    M Source/WebCore/platform/graphics/gstreamer/InbandMetadataTextTrackPrivateGStreamer.h
    M Source/WebCore/platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp
    M Source/WebCore/platform/graphics/gstreamer/TrackPrivateBaseGStreamer.cpp
    M Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp
    M Source/WebKit/GPUProcess/media/RemoteAudioTrackProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteAudioTrackProxy.h
    M Source/WebKit/GPUProcess/media/RemoteTextTrackProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteTextTrackProxy.h
    M Source/WebKit/GPUProcess/media/RemoteVideoTrackProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteVideoTrackProxy.h
    M Source/WebKit/GPUProcess/media/TextTrackPrivateRemoteConfiguration.h
    M Source/WebKit/GPUProcess/media/TextTrackPrivateRemoteConfiguration.serialization.in
    M Source/WebKit/GPUProcess/media/TrackPrivateRemoteConfiguration.h
    M Source/WebKit/GPUProcess/media/TrackPrivateRemoteConfiguration.serialization.in
    M Source/WebKit/WebProcess/GPU/media/AudioTrackPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/AudioTrackPrivateRemote.h
    M Source/WebKit/WebProcess/GPU/media/TextTrackPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/TextTrackPrivateRemote.h
    M Source/WebKit/WebProcess/GPU/media/VideoTrackPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/VideoTrackPrivateRemote.h

  Log Message:
  -----------
  [MSE] `ASSERT(canSafelyBeUsed());` ASSERTION when using webm with MSE in a worker
https://bugs.webkit.org/show_bug.cgi?id=269638
rdar://123134428

Reviewed by Eric Carlson.

With a MediaSource running in a DedicatedWorker, a TrackPrivate may be referenced by two Tracks:
- The canonical track living in the SourceBuffer
- Its mirror that is sent to the HTMLMediaElement on the main thread.

The interface between the Track and the TrackPrivate is done via the TrackPrivateBaseClient
which inherit from CanMakeWeakPtr. A WeakPtr was used to store the TrackPrivateClient
in the TrackPrivate.
As the TrackPrivate could only track one client, it was the last caller to TrackPrivate::setClient
that won. So you could end up having the TrackPrivate attempting to use the WeakPtr created on the
worker thread while the TrackPrivate is being used in the main thread.

We add support for multiple clients, and to ensure that the client will only ever be called
on the right thread, TrackPrivate::addClient now takes a dispatcher that will ensure the
task to run and the WeakPtr will always be accessed on the right thread.
TrackPrivate::addClient|removeClient are thread safe.

Previously, {Audio|Video|InbandTrack}PrivateTrack each managed their respective client.
We move the logic to the base TrackPrivateBase class to remove mostly duplicated code.

Move semantics with the client's callback is no longer possible, this is fine for
Audio and Video tracks as their message is only to enable/select them; however for
InbandTextTrack it becomes problematic as the WebVTT parser API and Cues all require
move semantics.
However, InbandTextTracks aren't currently supported with our MediaSource implementation
(no User-Agent does), so there will only ever be one client to those tracks, and the
client will be running in the main thread.
We add assertions for methods were moved semantics is required, and to prevent unexpected
failures, only the first client will be called, and only if on the main thread.

The Remote{Audio|Video|InbandText}PrivateTrack which are the only tracks used when MSE in
a worker is active, were using AtomString which can't be transferred across thread.
We change the type of their strings from AtomString to String, and ensure that
the string copied are isolated copies.

For non-MSE, no change in observable behaviour.
A test will be added in a follow-up change: https://webkit.org/b/269564)

* Source/WebCore/html/track/AudioTrack.cpp:
(WebCore::AudioTrack::AudioTrack):
(WebCore::AudioTrack::~AudioTrack):
(WebCore::AudioTrack::setPrivate):
* Source/WebCore/html/track/InbandTextTrack.cpp:
(WebCore::InbandTextTrack::InbandTextTrack):
(WebCore::InbandTextTrack::~InbandTextTrack):
(WebCore::InbandTextTrack::setPrivate):
* Source/WebCore/html/track/TrackBase.cpp:
(WebCore::TrackBase::addClientToTrackPrivateBase):
(WebCore::TrackBase::removeClientFromTrackPrivateBase):
* Source/WebCore/html/track/TrackBase.h:
* Source/WebCore/html/track/VideoTrack.cpp:
(WebCore::VideoTrack::VideoTrack):
(WebCore::VideoTrack::~VideoTrack):
(WebCore::VideoTrack::setPrivate):
* Source/WebCore/platform/graphics/AudioTrackPrivate.h:
(WebCore::AudioTrackPrivate::setEnabled):
(WebCore::AudioTrackPrivate::setConfiguration):
(WebCore::AudioTrackPrivate::setClient): Deleted.
(WebCore::AudioTrackPrivate::clearClient): Deleted.
* Source/WebCore/platform/graphics/AudioTrackPrivateClient.h:
(isType):
* Source/WebCore/platform/graphics/InbandTextTrackPrivate.h:
(WebCore::InbandTextTrackPrivate::setClient): Deleted.
(WebCore::InbandTextTrackPrivate::clearClient): Deleted.
* Source/WebCore/platform/graphics/InbandTextTrackPrivateClient.h:
(isType):
* Source/WebCore/platform/graphics/TrackPrivateBase.cpp:
(WebCore::TrackPrivateBase::notifyClients):
(WebCore::TrackPrivateBase::notifyMainThreadClient):
(WebCore::TrackPrivateBase::addClient):
(WebCore::TrackPrivateBase::removeClient):
(WebCore::TrackPrivateBase::hasClients const):
(WebCore::TrackPrivateBase::hasOneClient const):
* Source/WebCore/platform/graphics/TrackPrivateBase.h:
* Source/WebCore/platform/graphics/TrackPrivateBaseClient.h:
* Source/WebCore/platform/graphics/VideoTrackPrivate.h:
(WebCore::VideoTrackPrivate::setSelected):
(WebCore::VideoTrackPrivate::setConfiguration):
(WebCore::VideoTrackPrivate::setClient): Deleted.
(WebCore::VideoTrackPrivate::clearClient): Deleted.
* Source/WebCore/platform/graphics/VideoTrackPrivateClient.h:
(isType):
* Source/WebCore/platform/graphics/avfoundation/InbandMetadataTextTrackPrivateAVF.cpp:
(WebCore::InbandMetadataTextTrackPrivateAVF::addDataCue):
(WebCore::InbandMetadataTextTrackPrivateAVF::updatePendingCueEndTimes):
(WebCore::InbandMetadataTextTrackPrivateAVF::flushPartialCues):
* Source/WebCore/platform/graphics/avfoundation/InbandTextTrackPrivateAVF.cpp:
(WebCore::InbandTextTrackPrivateAVF::processCue):
(WebCore::InbandTextTrackPrivateAVF::processAttributedStrings):
(WebCore::InbandTextTrackPrivateAVF::removeCompletedCues):
(WebCore::InbandTextTrackPrivateAVF::resetCueValues):
(WebCore::InbandTextTrackPrivateAVF::processNativeSamples):
* Source/WebCore/platform/graphics/avfoundation/objc/InbandChapterTrackPrivateAVFObjC.mm:
(WebCore::InbandChapterTrackPrivateAVFObjC::processChapters):
* Source/WebCore/platform/graphics/gstreamer/AudioTrackPrivateGStreamer.cpp:
(WebCore::AudioTrackPrivateGStreamer::updateConfigurationFromTags):
* Source/WebCore/platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp:
(WebCore::InbandTextTrackPrivateGStreamer::tagsChanged):
(WebCore::InbandTextTrackPrivateGStreamer::notifyTrackOfSample):
* Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp:
(WebCore::VideoTrackPrivateGStreamer::updateConfigurationFromTags):
* Source/WebKit/GPUProcess/media/RemoteAudioTrackProxy.cpp:
(WebKit::RemoteAudioTrackProxy::RemoteAudioTrackProxy):
(WebKit::RemoteAudioTrackProxy::~RemoteAudioTrackProxy):
* Source/WebKit/GPUProcess/media/RemoteAudioTrackProxy.h:
* Source/WebKit/GPUProcess/media/RemoteTextTrackProxy.cpp:
(WebKit::RemoteTextTrackProxy::RemoteTextTrackProxy):
(WebKit::RemoteTextTrackProxy::~RemoteTextTrackProxy):
* Source/WebKit/GPUProcess/media/RemoteTextTrackProxy.h:
* Source/WebKit/GPUProcess/media/RemoteVideoTrackProxy.cpp:
(WebKit::RemoteVideoTrackProxy::RemoteVideoTrackProxy):
(WebKit::RemoteVideoTrackProxy::~RemoteVideoTrackProxy):
* Source/WebKit/GPUProcess/media/RemoteVideoTrackProxy.h:
* Source/WebKit/GPUProcess/media/TextTrackPrivateRemoteConfiguration.h:
* Source/WebKit/GPUProcess/media/TextTrackPrivateRemoteConfiguration.serialization.in: Change data type to AtomString
* Source/WebKit/GPUProcess/media/TrackPrivateRemoteConfiguration.h:
* Source/WebKit/GPUProcess/media/TrackPrivateRemoteConfiguration.serialization.in: Change data type to AtomString
* Source/WebKit/WebProcess/GPU/media/AudioTrackPrivateRemote.cpp:
(WebKit::AudioTrackPrivateRemote::updateConfiguration):
* Source/WebKit/WebProcess/GPU/media/AudioTrackPrivateRemote.h:
* Source/WebKit/WebProcess/GPU/media/TextTrackPrivateRemote.cpp:
(WebKit::TextTrackPrivateRemote::updateConfiguration):
(WebKit::TextTrackPrivateRemote::addGenericCue):
(WebKit::TextTrackPrivateRemote::updateGenericCue):
(WebKit::TextTrackPrivateRemote::removeGenericCue):
(WebKit::TextTrackPrivateRemote::parseWebVTTFileHeader):
(WebKit::TextTrackPrivateRemote::parseWebVTTCueData):
(WebKit::TextTrackPrivateRemote::parseWebVTTCueDataStruct):
(WebKit::TextTrackPrivateRemote::addDataCue):
(WebKit::TextTrackPrivateRemote::addDataCueWithType):
(WebKit::TextTrackPrivateRemote::updateDataCue):
(WebKit::TextTrackPrivateRemote::removeDataCue):
* Source/WebKit/WebProcess/GPU/media/TextTrackPrivateRemote.h:
* Source/WebKit/WebProcess/GPU/media/VideoTrackPrivateRemote.cpp:
(WebKit::VideoTrackPrivateRemote::updateConfiguration):
* Source/WebKit/WebProcess/GPU/media/VideoTrackPrivateRemote.h:

Canonical link: https://commits.webkit.org/274956@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications