[webkit-changes] [WebKit/WebKit] 0b9dfb: Implement MediaStreamTrack transfer to dedicated w...

[youennf]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 0b9dfbd9559968b03834522225f58dcaeffaa57f
      https://github.com/WebKit/WebKit/commit/0b9dfbd9559968b03834522225f58dcaeffaa57f
  Author: Youenn Fablet <youennf at gmail.com>
  Date:   2024-01-22 (Mon, 22 Jan 2024)

  Changed paths:
    A LayoutTests/http/wpt/mediastream/transfer-mediastreamtrack-to-worker-expected.txt
    A LayoutTests/http/wpt/mediastream/transfer-mediastreamtrack-to-worker-worker.js
    A LayoutTests/http/wpt/mediastream/transfer-mediastreamtrack-to-worker.html
    A LayoutTests/http/wpt/mediastream/transfer-videotrackgenerator-track-expected.txt
    A LayoutTests/http/wpt/mediastream/transfer-videotrackgenerator-track.html
    M LayoutTests/imported/w3c/web-platform-tests/mediacapture-insertable-streams/idlharness.any.worker-expected.txt
    M LayoutTests/imported/w3c/web-platform-tests/mediacapture-streams/MediaStreamTrack-transfer-video.https-expected.txt
    M LayoutTests/platform/glib/TestExpectations
    M LayoutTests/platform/mac-wk1/TestExpectations
    M Source/WebCore/Headers.cmake
    M Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp
    M Source/WebCore/Modules/mediastream/MediaStreamTrack.h
    M Source/WebCore/Modules/mediastream/MediaStreamTrack.idl
    M Source/WebCore/Modules/mediastream/MediaStreamTrackProcessor.cpp
    M Source/WebCore/Sources.txt
    M Source/WebCore/WebCore.xcodeproj/project.pbxproj
    M Source/WebCore/bindings/js/SerializedScriptValue.cpp
    M Source/WebCore/bindings/js/SerializedScriptValue.h
    A Source/WebCore/platform/mediastream/MediaStreamTrackDataHolder.cpp
    A Source/WebCore/platform/mediastream/MediaStreamTrackDataHolder.h
    M Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.cpp
    M Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.h
    M Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in
    M Tools/TestWebKitAPI/Tests/WebKit/GetUserMedia.mm
    M Tools/TestWebKitAPI/cocoa/UserMediaCaptureUIDelegate.h
    M Tools/TestWebKitAPI/cocoa/UserMediaCaptureUIDelegate.mm

  Log Message:
  -----------
  Implement MediaStreamTrack transfer to dedicated workers in same agent cluster
rdar://121180907
https://bugs.webkit.org/show_bug.cgi?id=267682

Reviewed by Eric Carlson.

Introduce MediaStreamTrackDataHolder which is used to store the MediaStreamTrack information necessary as part of the transfer.
It keeps the track source alive and allows to create a MediaStreamTrackPrivate from it.
We expose MediaStreamTrack to window and dedicated worker but only support transferring tracks in the same agent cluster.
The goal is typically to transfer the track from a window context to a dedicated worker created by the window context, to do track video frame processing.

Given the source, even if transferred, is kept by the document for which was created the source, there is no impact on the privacy indicators.

The new tests highlighted three issues in MediaStreamTrackProcessor:
- The observer needs to be created right away or we need to add an if check. We are creating it right away
- We were missing protecting the processor in MediaStreamTrackProcessor::VideoFrameObserver::videoFrameAvailable.
- We should call ReadableStreamSource::pullFinished() only if enqueuing went well.

* LayoutTests/http/wpt/mediastream/transfer-mediastreamtrack-to-worker-expected.txt: Added.
* LayoutTests/http/wpt/mediastream/transfer-mediastreamtrack-to-worker-worker.js: Added.
(makeOffscreenCanvasVideoFrame):
(promise_test.async t):
* LayoutTests/http/wpt/mediastream/transfer-mediastreamtrack-to-worker.html: Added.
* LayoutTests/http/wpt/mediastream/transfer-videotrackgenerator-track-expected.txt: Added.
* LayoutTests/http/wpt/mediastream/transfer-videotrackgenerator-track.html: Added.
* LayoutTests/imported/w3c/web-platform-tests/mediacapture-insertable-streams/idlharness.any.worker-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/mediacapture-streams/MediaStreamTrack-transfer-video.https-expected.txt:
* LayoutTests/platform/glib/TestExpectations:
* LayoutTests/platform/mac-wk1/TestExpectations:
* Source/WebCore/Headers.cmake:
* Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp:
(WebCore::MediaStreamTrack::detach):
(WebCore::MediaStreamTrack::create):
* Source/WebCore/Modules/mediastream/MediaStreamTrack.h:
(WebCore::MediaStreamTrack::isDetached const):
* Source/WebCore/Modules/mediastream/MediaStreamTrack.idl:
* Source/WebCore/Modules/mediastream/MediaStreamTrackProcessor.cpp:
(WebCore::MediaStreamTrackProcessor::VideoFrameObserverWrapper::initialize):
(WebCore::MediaStreamTrackProcessor::VideoFrameObserver::VideoFrameObserver):
(WebCore::MediaStreamTrackProcessor::VideoFrameObserver::videoFrameAvailable):
(WebCore::MediaStreamTrackProcessor::Source::enqueue):
* Source/WebCore/Sources.txt:
* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/bindings/js/SerializedScriptValue.cpp:
(WebCore::isTypeExposedToGlobalObject):
(WebCore::CloneSerializer::serialize):
(WebCore::CloneSerializer::CloneSerializer):
(WebCore::CloneSerializer::dumpMediaStreamTrack):
(WebCore::CloneSerializer::dumpIfTerminal):
(WebCore::CloneDeserializer::deserialize):
(WebCore::CloneDeserializer::readMediaStreamTrack):
(WebCore::CloneDeserializer::readTerminal):
(WebCore::SerializedScriptValue::SerializedScriptValue):
(WebCore::canDetachMediaStreamTracks):
(WebCore::SerializedScriptValue::create):
(WebCore::SerializedScriptValue::deserialize):
* Source/WebCore/bindings/js/SerializedScriptValue.h:
* Source/WebCore/platform/mediastream/MediaStreamTrackDataHolder.cpp: Added.
(WebCore::PreventSourceFromEndingObserverWrapper::create):
(WebCore::PreventSourceFromEndingObserverWrapper::initialize):
(WebCore::MediaStreamTrackDataHolder::MediaStreamTrackDataHolder):
(WebCore::MediaStreamTrackDataHolder::~MediaStreamTrackDataHolder):
* Source/WebCore/platform/mediastream/MediaStreamTrackDataHolder.h: Added.
* Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.cpp:
(WebCore::MediaStreamTrackPrivate::create):
(WebCore::MediaStreamTrackPrivate::MediaStreamTrackPrivate):
(WebCore::MediaStreamTrackPrivate::toDataHolder):
* Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.h:
* Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in:
* Tools/TestWebKitAPI/Tests/WebKit/GetUserMedia.mm:
(TestWebKitAPI::captureAndTransferAudioTrack):
(TestWebKitAPI::captureAndTransferVideoTrack):
(TestWebKitAPI::start):
* Tools/TestWebKitAPI/cocoa/UserMediaCaptureUIDelegate.h:
* Tools/TestWebKitAPI/cocoa/UserMediaCaptureUIDelegate.mm:
(-[UserMediaCaptureUIDelegate setWebViewForPopup:]):
(-[UserMediaCaptureUIDelegate webView:createWebViewWithConfiguration:forNavigationAction:windowFeatures:]):
(-[UserMediaCaptureUIDelegate webView:decidePolicyForNavigationAction:preferences:decisionHandler:]):

Canonical link: https://commits.webkit.org/273300@main