[webkit-changes] [WebKit/WebKit] b65f4e: Add a MediaStreamTrackProcessor skeleton and imple...

[youennf]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: b65f4ed6d7773e43c59f050351d01b7396f8e0d8
      https://github.com/WebKit/WebKit/commit/b65f4ed6d7773e43c59f050351d01b7396f8e0d8
  Author: Youenn Fablet <youennf at gmail.com>
  Date:   2024-01-17 (Wed, 17 Jan 2024)

  Changed paths:
    M LayoutTests/http/wpt/mediastream/worker-mediastreamtrack.worker-expected.txt
    M LayoutTests/http/wpt/mediastream/worker-mediastreamtrack.worker.js
    M LayoutTests/imported/w3c/web-platform-tests/mediacapture-insertable-streams/idlharness.any.js
    M LayoutTests/imported/w3c/web-platform-tests/mediacapture-insertable-streams/idlharness.any.worker-expected.txt
    M Source/WebCore/CMakeLists.txt
    M Source/WebCore/DerivedSources-input.xcfilelist
    M Source/WebCore/DerivedSources-output.xcfilelist
    M Source/WebCore/DerivedSources.make
    M Source/WebCore/Modules/mediastream/MediaStreamTrack.h
    A Source/WebCore/Modules/mediastream/MediaStreamTrackProcessor.cpp
    A Source/WebCore/Modules/mediastream/MediaStreamTrackProcessor.h
    A Source/WebCore/Modules/mediastream/MediaStreamTrackProcessor.idl
    M Source/WebCore/Modules/mediastream/VideoTrackGenerator.cpp
    M Source/WebCore/Modules/mediastream/VideoTrackGenerator.h
    M Source/WebCore/Modules/mediastream/VideoTrackGenerator.idl
    M Source/WebCore/Sources.txt
    M Source/WebCore/WebCore.xcodeproj/project.pbxproj
    M Source/WebCore/bindings/js/WebCoreBuiltinNames.h
    M Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.cpp
    M Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.h

  Log Message:
  -----------
  Add a MediaStreamTrackProcessor skeleton and implementation
https://bugs.webkit.org/show_bug.cgi?id=267317
rdar://120771789

Reviewed by Eric Carlson.

Introduce MediaStreamTrackProcessor by getting video frames from the track source and piping them to a ReadableStream source.
We get the video frame from the video pipeline, we store a video frame there and hop to the worker thread where we try to enqueue the video frame.

We use a buffer of one video frame for now.
While each new frame will trigger a task to the worker thread, video frames may currently be dropped if thr worker thread is not fast enough.
Similarly, video frames might be dropped if the web page is not consuming them.

Fix VideoTrackGenerator compile flag to include WEB_CODECS.

Covered by added tests.

* LayoutTests/http/wpt/mediastream/worker-mediastreamtrack.worker-expected.txt:
* LayoutTests/http/wpt/mediastream/worker-mediastreamtrack.worker.js:
(promise_test.async t):
* LayoutTests/imported/w3c/web-platform-tests/mediacapture-insertable-streams/idlharness.any.js:
* LayoutTests/imported/w3c/web-platform-tests/mediacapture-insertable-streams/idlharness.any.worker-expected.txt:
* Source/WebCore/CMakeLists.txt:
* Source/WebCore/DerivedSources-input.xcfilelist:
* Source/WebCore/DerivedSources-output.xcfilelist:
* Source/WebCore/DerivedSources.make:
* Source/WebCore/Modules/mediastream/MediaStreamTrack.h:
(WebCore::MediaStreamTrack::sourceForProcessor const):
* Source/WebCore/Modules/mediastream/MediaStreamTrackProcessor.cpp: Added.
(WebCore::MediaStreamTrackProcessor::create):
(WebCore::MediaStreamTrackProcessor::MediaStreamTrackProcessor):
(WebCore::m_track):
(WebCore::MediaStreamTrackProcessor::~MediaStreamTrackProcessor):
(WebCore::MediaStreamTrackProcessor::readable):
(WebCore::MediaStreamTrackProcessor::contextDestroyed):
(WebCore::MediaStreamTrackProcessor::stopVideoFrameObserver):
(WebCore::MediaStreamTrackProcessor::tryEnqueueingVideoFrame):
(WebCore::MediaStreamTrackProcessor::VideoFrameObserverWrapper::create):
(WebCore::MediaStreamTrackProcessor::VideoFrameObserverWrapper::VideoFrameObserverWrapper):
(WebCore::MediaStreamTrackProcessor::VideoFrameObserverWrapper::initialize):
(WebCore::MediaStreamTrackProcessor::VideoFrameObserverWrapper::start):
(WebCore::MediaStreamTrackProcessor::VideoFrameObserver::VideoFrameObserver):
(WebCore::MediaStreamTrackProcessor::VideoFrameObserver::start):
(WebCore::MediaStreamTrackProcessor::VideoFrameObserver::~VideoFrameObserver):
(WebCore::MediaStreamTrackProcessor::VideoFrameObserver::takeVideoFrame):
(WebCore::MediaStreamTrackProcessor::VideoFrameObserver::videoFrameAvailable):
(WebCore::MediaStreamTrackProcessor::Source::Source):
(WebCore::MediaStreamTrackProcessor::Source::~Source):
(WebCore::MediaStreamTrackProcessor::Source::trackEnded):
(WebCore::MediaStreamTrackProcessor::Source::isWaiting const):
(WebCore::MediaStreamTrackProcessor::Source::close):
(WebCore::MediaStreamTrackProcessor::Source::enqueue):
(WebCore::MediaStreamTrackProcessor::Source::doStart):
(WebCore::MediaStreamTrackProcessor::Source::doPull):
(WebCore::MediaStreamTrackProcessor::Source::doCancel):
* Source/WebCore/Modules/mediastream/MediaStreamTrackProcessor.h: Added.
(WebCore::MediaStreamTrackProcessor::VideoFrameObserverWrapper::takeVideoFrame):
* Source/WebCore/Modules/mediastream/MediaStreamTrackProcessor.idl: Copied from Source/WebCore/Modules/mediastream/VideoTrackGenerator.idl.
* Source/WebCore/Modules/mediastream/VideoTrackGenerator.cpp:
* Source/WebCore/Modules/mediastream/VideoTrackGenerator.h:
* Source/WebCore/Modules/mediastream/VideoTrackGenerator.idl:
* Source/WebCore/Sources.txt:
* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/bindings/js/WebCoreBuiltinNames.h:
* Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.cpp:
(WebCore::MediaStreamTrackPrivate::sourceForProcessor):
* Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.h:

Canonical link: https://commits.webkit.org/273117@main