[webkit-changes] [WebKit/WebKit] c6f463: [GStreamer][WebCodecs] Video encoding and decoding...

[Philippe Normand]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: c6f463099606c3baba97dbc40b336aecaf495fd3
      https://github.com/WebKit/WebKit/commit/c6f463099606c3baba97dbc40b336aecaf495fd3
  Author: Philippe Normand <philn at igalia.com>
  Date:   2023-01-16 (Mon, 16 Jan 2023)

  Changed paths:
    M LayoutTests/platform/glib/TestExpectations
    A LayoutTests/platform/glib/imported/w3c/web-platform-tests/webcodecs/temporal-svc-encoding.https.any.worker_h264-expected.txt
    A LayoutTests/platform/glib/imported/w3c/web-platform-tests/webcodecs/temporal-svc-encoding.https.any.worker_vp8-expected.txt
    A LayoutTests/platform/glib/imported/w3c/web-platform-tests/webcodecs/temporal-svc-encoding.https.any.worker_vp9-expected.txt
    A LayoutTests/platform/glib/imported/w3c/web-platform-tests/webcodecs/videoFrame-createImageBitmap.any-expected.txt
    A LayoutTests/platform/glib/imported/w3c/web-platform-tests/webcodecs/videoFrame-drawImage.any-expected.txt
    M Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml
    M Source/WebCore/Modules/webcodecs/WebCodecsVideoFrame.cpp
    M Source/WebCore/platform/GStreamer.cmake
    M Source/WebCore/platform/VideoDecoder.cpp
    M Source/WebCore/platform/VideoEncoder.cpp
    M Source/WebCore/platform/VideoFrame.cpp
    A Source/WebCore/platform/graphics/gstreamer/VideoDecoderGStreamer.cpp
    A Source/WebCore/platform/graphics/gstreamer/VideoDecoderGStreamer.h
    A Source/WebCore/platform/graphics/gstreamer/VideoEncoderGStreamer.cpp
    A Source/WebCore/platform/graphics/gstreamer/VideoEncoderGStreamer.h
    M Source/WebCore/platform/graphics/gstreamer/VideoFrameGStreamer.cpp
    M Source/WebCore/platform/graphics/gstreamer/VideoFrameGStreamer.h
    A Source/WebCore/platform/gstreamer/GStreamerElementHarness.cpp
    A Source/WebCore/platform/gstreamer/GStreamerElementHarness.h
    M Source/cmake/OptionsGTK.cmake
    M Source/cmake/OptionsWPE.cmake
    M Tools/TestWebKitAPI/PlatformGTK.cmake
    M Tools/TestWebKitAPI/PlatformWPE.cmake
    A Tools/TestWebKitAPI/Tests/WebCore/gstreamer/GstElementHarness.cpp

  Log Message:
  -----------
  [GStreamer][WebCodecs] Video encoding and decoding support
https://bugs.webkit.org/show_bug.cgi?id=249972

Reviewed by Xabier Rodriguez-Calvar.

The video encoder makes use of the WebRTC encoder. The video decoder relies on the platform decoder
previously probed through the registry scanner. In order to have integrate the encoding and decoding
at the lowest level possible we make use of a new component called GstElementHarness, so we have
full control on the input and output workflow of the encoder and decoder.

* LayoutTests/platform/glib/TestExpectations:
* LayoutTests/platform/glib/imported/w3c/web-platform-tests/webcodecs/temporal-svc-encoding.https.any.worker_h264-expected.txt: Added.
* LayoutTests/platform/glib/imported/w3c/web-platform-tests/webcodecs/temporal-svc-encoding.https.any.worker_vp8-expected.txt: Added.
* LayoutTests/platform/glib/imported/w3c/web-platform-tests/webcodecs/temporal-svc-encoding.https.any.worker_vp9-expected.txt: Added.
* LayoutTests/platform/glib/imported/w3c/web-platform-tests/webcodecs/videoFrame-createImageBitmap.any-expected.txt: Added.
* LayoutTests/platform/glib/imported/w3c/web-platform-tests/webcodecs/videoFrame-drawImage.any-expected.txt: Added.
* Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml:
* Source/WebCore/Modules/webcodecs/WebCodecsVideoFrame.cpp:
(WebCore::WebCodecsVideoFrame::create):
* Source/WebCore/platform/GStreamer.cmake:
* Source/WebCore/platform/VideoDecoder.cpp:
(WebCore::VideoDecoder::createLocalDecoder):
* Source/WebCore/platform/VideoEncoder.cpp:
(WebCore::VideoEncoder::createLocalEncoder):
* Source/WebCore/platform/VideoFrame.cpp:
(WebCore::VideoFrame::paintInContext):
* Source/WebCore/platform/graphics/gstreamer/VideoDecoderGStreamer.cpp: Added.
(WebCore::gstWorkQueue):
(WebCore::GStreamerInternalVideoDecoder::create):
(WebCore::GStreamerInternalVideoDecoder::postTask):
(WebCore::GStreamerInternalVideoDecoder::close):
(WebCore::GStreamerVideoDecoder::create):
(WebCore::GStreamerVideoDecoder::GStreamerVideoDecoder):
(WebCore::GStreamerVideoDecoder::~GStreamerVideoDecoder):
(WebCore::GStreamerVideoDecoder::decode):
(WebCore::GStreamerVideoDecoder::flush):
(WebCore::GStreamerVideoDecoder::reset):
(WebCore::GStreamerVideoDecoder::close):
(WebCore::GStreamerInternalVideoDecoder::GStreamerInternalVideoDecoder):
(WebCore::GStreamerInternalVideoDecoder::decode):
(WebCore::GStreamerInternalVideoDecoder::flush):
* Source/WebCore/platform/graphics/gstreamer/VideoDecoderGStreamer.h: Added.
* Source/WebCore/platform/graphics/gstreamer/VideoEncoderGStreamer.cpp: Added.
(WebCore::gstWorkQueue):
(WebCore::GStreamerInternalVideoEncoder::create):
(WebCore::GStreamerInternalVideoEncoder::postTask):
(WebCore::GStreamerInternalVideoEncoder::close):
(WebCore::GStreamerInternalVideoEncoder::harness const):
(WebCore::GStreamerInternalVideoEncoder::isClosed const):
(WebCore::GStreamerVideoEncoder::create):
(WebCore::GStreamerVideoEncoder::GStreamerVideoEncoder):
(WebCore::GStreamerVideoEncoder::~GStreamerVideoEncoder):
(WebCore::GStreamerVideoEncoder::initialize):
(WebCore::GStreamerVideoEncoder::encode):
(WebCore::GStreamerVideoEncoder::flush):
(WebCore::GStreamerVideoEncoder::reset):
(WebCore::GStreamerVideoEncoder::close):
(WebCore::GStreamerInternalVideoEncoder::GStreamerInternalVideoEncoder):
(WebCore::GStreamerInternalVideoEncoder::initialize):
(WebCore::GStreamerInternalVideoEncoder::encode):
(WebCore::GStreamerInternalVideoEncoder::flush):
* Source/WebCore/platform/graphics/gstreamer/VideoEncoderGStreamer.h: Added.
* Source/WebCore/platform/graphics/gstreamer/VideoFrameGStreamer.cpp:
(WebCore::GstSampleColorConverter::convertSample):
(WebCore::GstSampleColorConverter::convertSampleToImage):
(WebCore::VideoFrame::fromNativeImage):
(WebCore::copyToGstBufferPlane):
(WebCore::VideoFrame::createNV12):
(WebCore::VideoFrame::createRGBA):
(WebCore::VideoFrame::createBGRA):
(WebCore::VideoFrame::createI420):
(WebCore::VideoFrameGStreamer::create):
(WebCore::VideoFrameGStreamer::createWrappedSample):
(WebCore::VideoFrameGStreamer::VideoFrameGStreamer):
(WebCore::copyPlane):
(WebCore::VideoFrame::copyTo):
(WebCore::VideoFrame::paintInContext):
* Source/WebCore/platform/graphics/gstreamer/VideoFrameGStreamer.h:
* Source/WebCore/platform/gstreamer/GStreamerElementHarness.cpp: Added.
(WebCore::GStreamerElementHarness::GStreamerElementHarness):
(WebCore::GStreamerElementHarness::~GStreamerElementHarness):
(WebCore::GStreamerElementHarness::start):
(WebCore::GStreamerElementHarness::pushStickyEvents):
(WebCore::GStreamerElementHarness::pushSample):
(WebCore::GStreamerElementHarness::pushBuffer):
(WebCore::GStreamerElementHarness::pushEvent):
(WebCore::GStreamerElementHarness::pullBuffer):
(WebCore::GStreamerElementHarness::pullEvent):
(WebCore::GStreamerElementHarness::outputCaps):
(WebCore::GStreamerElementHarness::chainBuffer):
(WebCore::GStreamerElementHarness::sinkQuery):
(WebCore::GStreamerElementHarness::sinkEvent):
(WebCore::GStreamerElementHarness::srcQuery):
(WebCore::GStreamerElementHarness::srcEvent):
(WebCore::GStreamerElementHarness::processOutputBuffers):
(WebCore::GStreamerElementHarness::flush):
* Source/WebCore/platform/gstreamer/GStreamerElementHarness.h: Added.
(WebCore::GStreamerElementHarness::create):
(WebCore::GStreamerElementHarness::element const):
* Source/cmake/OptionsGTK.cmake:
* Source/cmake/OptionsWPE.cmake:
* Tools/TestWebKitAPI/PlatformGTK.cmake:
* Tools/TestWebKitAPI/PlatformWPE.cmake:
* Tools/TestWebKitAPI/Tests/WebCore/gstreamer/GstElementHarness.cpp: Added.
(TestWebKitAPI::TEST_F):

Canonical link: https://commits.webkit.org/258949@main