[webkit-changes] [WebKit/WebKit] 9e81b9: Allow MediaStreamTrackPrivate to work out of main ...

[youennf]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 9e81b99af3104473041f6334bc92810d5c66da8f
      https://github.com/WebKit/WebKit/commit/9e81b99af3104473041f6334bc92810d5c66da8f
  Author: Youenn Fablet <youennf at gmail.com>
  Date:   2024-01-10 (Wed, 10 Jan 2024)

  Changed paths:
    M Source/WebCore/platform/mediarecorder/MediaRecorderPrivate.cpp
    M Source/WebCore/platform/mediastream/MediaConstraints.cpp
    M Source/WebCore/platform/mediastream/MediaConstraints.h
    M Source/WebCore/platform/mediastream/MediaStreamPrivate.cpp
    M Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.cpp
    M Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.h
    M Source/WebCore/platform/mediastream/RealtimeIncomingVideoSource.cpp
    M Source/WebCore/platform/mediastream/RealtimeIncomingVideoSource.h
    M Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp
    M Source/WebCore/platform/mediastream/RealtimeMediaSource.h
    M Source/WebCore/platform/mediastream/RealtimeMediaSourceCapabilities.h
    M Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.h
    M Source/WebCore/platform/mediastream/RealtimeMediaSourceSettings.cpp
    M Source/WebCore/platform/mediastream/RealtimeMediaSourceSettings.h
    M Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.cpp
    M Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp
    M Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in
    M Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp
    M Source/WebKit/WebProcess/cocoa/RemoteRealtimeVideoSource.cpp

  Log Message:
  -----------
  Allow MediaStreamTrackPrivate to work out of main thread
https://bugs.webkit.org/show_bug.cgi?id=267230
rdar://120652619

Reviewed by Eric Carlson.

We want to allow MediaStreamTrack to live in worker contexts, to allow MediaStreamTrackProcessor et al.
We do a refactoring of MediaStreamTrackPrivate so that it can handle a RealtimeMediaSource even though interaction is happening in a worker.
We keep the RealtimeMediaSource model (everything is main thread, except for media data delivery).
We introduce MediaStreamTrackPrivateSourceObserverWrapper which will observe the RealtimeMediaSource so that:
- It hops to main thread whenever interacting with the source (for instance for starting or muting a source).
- It hops back to MediaStreamTrackPrivate thread when being notified by RealtimeMediaSource.

We thus have to store more state in MediaStreamTrackPrivate (muted, settings and so on) and we update these states based on RealtimeMediaSource observing callbacks.
We also start to update the code to more directly use track/privateTrack instead of source wherever possible to better isolate sources from JS code.

We update RealtimeIncomingVideoSource so that clients are notified of settings that change.
This was the case for width and height but not frame rate.

Drive-by fix in UserMediaCaptureManagerProxy::SourceProxy where we should grab fresh settings whenever applyConstraints succeeds.

* Source/WebCore/platform/mediarecorder/MediaRecorderPrivate.cpp:
(WebCore::MediaRecorderPrivate::checkTrackState):
* Source/WebCore/platform/mediastream/MediaConstraints.cpp:
(WebCore::StringConstraint::isolatedCopy const):
(WebCore::MediaTrackConstraintSetMap::isolatedCopy const):
(WebCore::MediaConstraints::isolatedCopy const):
* Source/WebCore/platform/mediastream/MediaConstraints.h:
(WebCore::MediaTrackConstraintSetMap::MediaTrackConstraintSetMap):
* Source/WebCore/platform/mediastream/MediaStreamPrivate.cpp:
(WebCore::MediaStreamPrivate::monitorOrientation):
* Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.cpp:
(WebCore::MediaStreamTrackPrivate::create):
(WebCore::MediaStreamTrackPrivateSourceObserverWrapper::create):
(WebCore::MediaStreamTrackPrivateSourceObserverWrapper::initialize):
(WebCore::MediaStreamTrackPrivateSourceObserverWrapper::std::function<void):
(WebCore::MediaStreamTrackPrivateSourceObserverWrapper::source):
(WebCore::MediaStreamTrackPrivateSourceObserverWrapper::start):
(WebCore::MediaStreamTrackPrivateSourceObserverWrapper::stop):
(WebCore::MediaStreamTrackPrivateSourceObserverWrapper::requestToEnd):
(WebCore::MediaStreamTrackPrivateSourceObserverWrapper::setMuted):
(WebCore::MediaStreamTrackPrivateSourceObserverWrapper::close):
(WebCore::MediaStreamTrackPrivateSourceObserverWrapper::applyConstraints):
(WebCore::MediaStreamTrackPrivateSourceObserverWrapper::MediaStreamTrackPrivateSourceObserverWrapper):
(WebCore::MediaStreamTrackPrivate::MediaStreamTrackPrivate):
(WebCore::MediaStreamTrackPrivate::initialize):
(WebCore::MediaStreamTrackPrivate::~MediaStreamTrackPrivate):
(WebCore::MediaStreamTrackPrivate::isOnCreationThread):
(WebCore::MediaStreamTrackPrivate::forEachObserver):
(WebCore::MediaStreamTrackPrivate::addObserver):
(WebCore::MediaStreamTrackPrivate::removeObserver):
(WebCore::MediaStreamTrackPrivate::setContentHint):
(WebCore::MediaStreamTrackPrivate::startProducingData):
(WebCore::MediaStreamTrackPrivate::stopProducingData):
(WebCore::MediaStreamTrackPrivate::setIsInBackground):
(WebCore::MediaStreamTrackPrivate::setMuted):
(WebCore::MediaStreamTrackPrivate::setEnabled):
(WebCore::MediaStreamTrackPrivate::endTrack):
(WebCore::MediaStreamTrackPrivate::clone):
(WebCore::MediaStreamTrackPrivate::source):
(WebCore::MediaStreamTrackPrivate::hasSource const):
(WebCore::MediaStreamTrackPrivate::getPhotoCapabilities):
(WebCore::MediaStreamTrackPrivate::getPhotoSettings):
(WebCore::MediaStreamTrackPrivate::takePhoto):
(WebCore::MediaStreamTrackPrivate::applyConstraints):
(WebCore::MediaStreamTrackPrivate::createAudioSourceProvider):
(WebCore::MediaStreamTrackPrivate::sourceStarted):
(WebCore::MediaStreamTrackPrivate::sourceStopped):
(WebCore::MediaStreamTrackPrivate::sourceMutedChanged):
(WebCore::MediaStreamTrackPrivate::sourceSettingsChanged):
(WebCore::MediaStreamTrackPrivate::sourceConfigurationChanged):
(WebCore::MediaStreamTrackPrivate::hasStartedProducingData):
(WebCore::MediaStreamTrackPrivate::updateReadyState):
(WebCore::MediaStreamTrackPrivate::label const): Deleted.
(WebCore::MediaStreamTrackPrivate::muted const): Deleted.
(WebCore::MediaStreamTrackPrivate::interrupted const): Deleted.
(WebCore::MediaStreamTrackPrivate::isCaptureTrack const): Deleted.
(WebCore::MediaStreamTrackPrivate::settings const): Deleted.
(WebCore::MediaStreamTrackPrivate::capabilities const): Deleted.
(WebCore::MediaStreamTrackPrivate::preventSourceFromEnding): Deleted.
(WebCore::MediaStreamTrackPrivate::audioUnitWillStart): Deleted.
* Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.h:
* Source/WebCore/platform/mediastream/RealtimeIncomingVideoSource.cpp:
(WebCore::RealtimeIncomingVideoSource::settings):
(WebCore::RealtimeIncomingVideoSource::settingsDidChange):
(WebCore::RealtimeIncomingVideoSource::notifyNewFrame):
* Source/WebCore/platform/mediastream/RealtimeIncomingVideoSource.h:
* Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp:
(WebCore::RealtimeMediaSource::initializePersistentId):
(WebCore::RealtimeMediaSource::hashedId const):
* Source/WebCore/platform/mediastream/RealtimeMediaSource.h:
(WebCore::RealtimeMediaSource::setName):
(WebCore::RealtimeMediaSource::isVideoSource const): Deleted.
* Source/WebCore/platform/mediastream/RealtimeMediaSourceCapabilities.h:
(WebCore::RealtimeMediaSourceCapabilities::RealtimeMediaSourceCapabilities):
(WebCore::RealtimeMediaSourceCapabilities::deviceId const):
(WebCore::RealtimeMediaSourceCapabilities::setDeviceId):
(WebCore::RealtimeMediaSourceCapabilities::groupId const):
(WebCore::RealtimeMediaSourceCapabilities::setGroupId):
(WebCore::RealtimeMediaSourceCapabilities::isolatedCopy const):
* Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.h:
* Source/WebCore/platform/mediastream/RealtimeMediaSourceSettings.cpp:
(WebCore::RealtimeMediaSourceSettings::isolatedCopy const):
* Source/WebCore/platform/mediastream/RealtimeMediaSourceSettings.h:
(WebCore::RealtimeMediaSourceSettings::RealtimeMediaSourceSettings):
(WebCore::RealtimeMediaSourceSettings::deviceId const):
(WebCore::RealtimeMediaSourceSettings::setDeviceId):
(WebCore::RealtimeMediaSourceSettings::label const):
(WebCore::RealtimeMediaSourceSettings::setLabel):
* Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.cpp:
(WebCore::CoreAudioCaptureSource::handleNewCurrentMicrophoneDevice):
* Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp:
(WebCore::MockRealtimeVideoSource::drawText):
* Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in:
* Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp:
(WebKit::UserMediaCaptureManagerProxy::SourceProxy::applyConstraints):
* Source/WebKit/WebProcess/cocoa/RemoteRealtimeVideoSource.cpp:
(WebKit::RemoteRealtimeVideoSource::clone):

Canonical link: https://commits.webkit.org/272844@main