[webkit-changes] [WebKit/WebKit] e0a0bc: [GPUP] Remove TrackPrivateRemoteIdentifier,

Wed Dec 6 01:52:15 PST 2023

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: e0a0bc967262c6e4741e8539ab3c32c3bc310233
      https://github.com/WebKit/WebKit/commit/e0a0bc967262c6e4741e8539ab3c32c3bc310233
  Author: Jean-Yves Avenard <jya at apple.com>
  Date:   2023-12-06 (Wed, 06 Dec 2023)

  Changed paths:
    A LayoutTests/media/content/short--1-track.webm
    A LayoutTests/media/media-audio-track-expected.txt
    A LayoutTests/media/media-audio-track.html
    M LayoutTests/platform/glib/TestExpectations
    M LayoutTests/platform/mac-wk1/TestExpectations
    M Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp
    M Source/WebCore/platform/graphics/avfoundation/objc/InbandChapterTrackPrivateAVFObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/InbandChapterTrackPrivateAVFObjC.mm
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm
    M Source/WebKit/GPUProcess/media/InitializationSegmentInfo.h
    M Source/WebKit/GPUProcess/media/InitializationSegmentInfo.serialization.in
    M Source/WebKit/GPUProcess/media/RemoteAudioTrackProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteAudioTrackProxy.h
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerManagerProxy.h
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in
    M Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.h
    M Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.messages.in
    M Source/WebKit/GPUProcess/media/RemoteTextTrackProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteTextTrackProxy.h
    M Source/WebKit/GPUProcess/media/RemoteVideoTrackProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteVideoTrackProxy.h
    M Source/WebKit/GPUProcess/media/TextTrackPrivateRemoteConfiguration.serialization.in
    M Source/WebKit/GPUProcess/media/TrackPrivateRemoteConfiguration.h
    M Source/WebKit/GPUProcess/media/TrackPrivateRemoteConfiguration.serialization.in
    R Source/WebKit/GPUProcess/media/TrackPrivateRemoteIdentifier.h
    M Source/WebKit/Scripts/webkit/messages.py
    M Source/WebKit/Scripts/webkit/tests/MessageArgumentDescriptions.cpp
    M Source/WebKit/Shared/WTFArgumentCoders.serialization.in
    M Source/WebKit/WebKit.xcodeproj/project.pbxproj
    M Source/WebKit/WebProcess/GPU/media/AudioTrackPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/AudioTrackPrivateRemote.h
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.messages.in
    M Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.h
    M Source/WebKit/WebProcess/GPU/media/TextTrackPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/TextTrackPrivateRemote.h
    M Source/WebKit/WebProcess/GPU/media/VideoTrackPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/VideoTrackPrivateRemote.h

  Log Message:
  -----------
  [GPUP] Remove TrackPrivateRemoteIdentifier,
https://bugs.webkit.org/show_bug.cgi?id=265718
rdar://119070875

Reviewed by Jer Noble.

With TrackPrivate now using an uint64_t as track identifier,
we no longer require the TrackPrivateRemoteIdentifier object
to bridge the old AtomString trackID between the content and GPU process.
Instead we can use the TrackID int everywhere. The TrackID comes from the
media container that originated from the content process but was demuxed in the
GPU process.

We now fully comply with the HTML spec as giving access to the Audio/Video/Text
track to the container's track ID is a compulsory requirement:
"The user agent MUST source attribute values for id, kind, label and language for AudioTrack, VideoTrack and TextTrack objects as described for MPEG-4 ISOBMFF in the in-band tracks spec" [1][2]
That the ID of an Audio and Video track is an int is also a requirement:
ISO-BMFF "the decimal representation of the track_ID of a TrackHeaderBox (tkhd) in a TrackBox (trak)." [3]
WebM "Decimal representation of the TrackNumber field of the track in the Track section of the WebM file Segment" [4]
MPEG-TS: "Decimal representation of the elementary stream's identifier (elementary_PID field) in the PMT." [5]

In ISO-BMFF, the track_id in a TrackHeaderBox is an unsigned 32 bits int.
In Mpeg-TS, the Elementary PID is a 13 bits unsigned int
In WebM, there is theoretically no limitation to the length of an int as the EBML design allows for variable length int.
However, Matroska does requires integers to be 0-8 bytes in size.
"EBML files use integers of variable size. This way, the file format doesn’t waste space with storing 32 or even 64 bit integers in placed where they might sometimes occur. The way the size is coded is inspired by the UTF-8 encoding format. [...] The Matroska file format does not allow integer lengths greater than 8, meaning that the number of leading zeros is not higher than 7 and that the total length can always be retrieved from the first byte." [6]

As a consequence, there is no theoretical restriction to the values usable by a TrackID,
making the HashTable unsuitable as storage container as it excludes at least two values
through the design of its hashing algorithm (it requires a value used for an empty element and a deleted one)
So we use WTF::StdUnorderedMap instead.
Using a StdUnorderedMap also provides the advantage that there is no longer a need to validate
the TrackID value prior sending it over IPC as there are no exclusions possible for its use as
key in a HashMap
Should the value not be linked to an existing TrackPrivate it will be ignored.

[1] https://w3c.github.io/mse-byte-stream-format-isobmff/#iso-init-segments
[2] https://w3c.github.io/mse-byte-stream-format-webm/#webm-init-segmentsA
[3] https://dev.w3.org/html5/html-sourcing-inband-tracks/#mpeg4
[4] https://dev.w3.org/html5/html-sourcing-inband-tracks/#webmA
[5] https://dev.w3.org/html5/html-sourcing-inband-tracks/#mpeg2ts
[6] https://www.matroska.org/files/matroska_file_format_alexander_noe.pdf

Fly-By: Fix the content process RemoteTrackPrivate that would never be deleted
when its corresponding TrackPrivateProxy was deleted in the GPU process.
Fly-By: InbandChapterTrackPrivateAVFObjC didn't have a unique TrackID, we assign
one to it. The code worked with the GPU process as the RemoteTrackPrivate
would have assigned it a unique ID; however following changes in 271419 at main,
the original track requires a unique TrackID as the RemoteTrackPrivate reports
the same.

Add test ensuring track ID of value 2^64-1 are playable.

* LayoutTests/media/content/short--1-track.webm: Added.
* LayoutTests/media/media-audio-track-expected.txt: Added.
* LayoutTests/media/media-audio-track.html: Added.
* Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp:
(WebCore::MediaPlayerPrivateAVFoundation::processNewAndRemovedTextTracks):
(WebCore::InbandChapterTrackPrivateAVFObjC::create): Add TrackID constructor argument
* Source/WebCore/platform/graphics/avfoundation/objc/InbandChapterTrackPrivateAVFObjC.mm:
(WebCore::InbandChapterTrackPrivateAVFObjC::InbandChapterTrackPrivateAVFObjC):
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
Assign a different TrackID whenever a text track is created.
(WebCore::MediaPlayerPrivateAVFoundationObjC::processChapterTracks):
* Source/WebKit/GPUProcess/media/InitializationSegmentInfo.h:
* Source/WebKit/GPUProcess/media/InitializationSegmentInfo.serialization.in:
* Source/WebKit/GPUProcess/media/RemoteAudioTrackProxy.cpp:
(WebKit::RemoteAudioTrackProxy::RemoteAudioTrackProxy):
(WebKit::RemoteAudioTrackProxy::configurationChanged):
* Source/WebKit/GPUProcess/media/RemoteAudioTrackProxy.h:
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerManagerProxy.h:
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp:
(WebKit::RemoteMediaPlayerProxy::addRemoteAudioTrackProxy):
(WebKit::RemoteMediaPlayerProxy::audioTrackSetEnabled):
(WebKit::RemoteMediaPlayerProxy::addRemoteVideoTrackProxy):
(WebKit::RemoteMediaPlayerProxy::videoTrackSetSelected):
(WebKit::RemoteMediaPlayerProxy::addRemoteTextTrackProxy):
(WebKit::RemoteMediaPlayerProxy::textTrackSetMode):
(WebKit::RemoteMediaPlayerProxy::mediaPlayerDidRemoveAudioTrack):
(WebKit::RemoteMediaPlayerProxy::mediaPlayerDidRemoveVideoTrack):
(WebKit::RemoteMediaPlayerProxy::mediaPlayerDidRemoveTextTrack):
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h:
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
* Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.cpp:
(WebKit::RemoteSourceBufferProxy::sourceBufferPrivateDidReceiveInitializationSegment):
(WebKit::RemoteSourceBufferProxy::addTrackBuffer):
(WebKit::RemoteSourceBufferProxy::updateTrackIds):
(WebKit::RemoteSourceBufferProxy::bufferedSamplesForTrackId):
(WebKit::RemoteSourceBufferProxy::enqueuedSamplesForTrackID):
* Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.h:
* Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.messages.in:
* Source/WebKit/GPUProcess/media/RemoteTextTrackProxy.cpp:
(WebKit::RemoteTextTrackProxy::RemoteTextTrackProxy):
(WebKit::RemoteTextTrackProxy::configurationChanged):
(WebKit::RemoteTextTrackProxy::addDataCue):
(WebKit::RemoteTextTrackProxy::updateDataCue):
(WebKit::RemoteTextTrackProxy::removeDataCue):
(WebKit::RemoteTextTrackProxy::addGenericCue):
(WebKit::RemoteTextTrackProxy::updateGenericCue):
(WebKit::RemoteTextTrackProxy::removeGenericCue):
(WebKit::RemoteTextTrackProxy::parseWebVTTFileHeader):
(WebKit::RemoteTextTrackProxy::parseWebVTTCueData):
* Source/WebKit/GPUProcess/media/RemoteTextTrackProxy.h:
* Source/WebKit/GPUProcess/media/RemoteVideoTrackProxy.cpp:
(WebKit::RemoteVideoTrackProxy::RemoteVideoTrackProxy):
(WebKit::RemoteVideoTrackProxy::updateConfiguration):
* Source/WebKit/GPUProcess/media/RemoteVideoTrackProxy.h:
* Source/WebKit/GPUProcess/media/TextTrackPrivateRemoteConfiguration.serialization.in:
* Source/WebKit/GPUProcess/media/TrackPrivateRemoteConfiguration.h:
* Source/WebKit/GPUProcess/media/TrackPrivateRemoteConfiguration.serialization.in:
* Source/WebKit/GPUProcess/media/TrackPrivateRemoteIdentifier.h: Removed.
* Source/WebKit/Scripts/webkit/messages.py:
(function_parameter_type):
(serialized_identifiers):
(types_that_cannot_be_forward_declared):
(headers_for_type):
* Source/WebKit/Scripts/webkit/tests/MessageArgumentDescriptions.cpp:
(IPC::serializedIdentifiers):
* Source/WebKit/Shared/WTFArgumentCoders.serialization.in:
* Source/WebKit/WebKit.xcodeproj/project.pbxproj:
* Source/WebKit/WebProcess/GPU/media/AudioTrackPrivateRemote.cpp:
(WebKit::AudioTrackPrivateRemote::AudioTrackPrivateRemote):
(WebKit::AudioTrackPrivateRemote::setEnabled):
* Source/WebKit/WebProcess/GPU/media/AudioTrackPrivateRemote.h:
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
(WebKit::MediaPlayerPrivateRemote::addRemoteAudioTrack):
(WebKit::MediaPlayerPrivateRemote::removeRemoteAudioTrack):
(WebKit::MediaPlayerPrivateRemote::remoteAudioTrackConfigurationChanged):
(WebKit::MediaPlayerPrivateRemote::addRemoteTextTrack):
(WebKit::MediaPlayerPrivateRemote::removeRemoteTextTrack):
(WebKit::MediaPlayerPrivateRemote::remoteTextTrackConfigurationChanged):
(WebKit::MediaPlayerPrivateRemote::parseWebVTTFileHeader):
(WebKit::MediaPlayerPrivateRemote::parseWebVTTCueData):
(WebKit::MediaPlayerPrivateRemote::parseWebVTTCueDataStruct):
(WebKit::MediaPlayerPrivateRemote::addDataCue):
(WebKit::MediaPlayerPrivateRemote::addDataCueWithType):
(WebKit::MediaPlayerPrivateRemote::updateDataCue):
(WebKit::MediaPlayerPrivateRemote::removeDataCue):
(WebKit::MediaPlayerPrivateRemote::addGenericCue):
(WebKit::MediaPlayerPrivateRemote::updateGenericCue):
(WebKit::MediaPlayerPrivateRemote::removeGenericCue):
(WebKit::MediaPlayerPrivateRemote::addRemoteVideoTrack):
(WebKit::MediaPlayerPrivateRemote::removeRemoteVideoTrack):
(WebKit::MediaPlayerPrivateRemote::remoteVideoTrackConfigurationChanged):
(WebKit::MediaPlayerPrivateRemote::audioTrackPrivateRemote const):
(WebKit::MediaPlayerPrivateRemote::videoTrackPrivateRemote const):
(WebKit::MediaPlayerPrivateRemote::textTrackPrivateRemote const):
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.messages.in:
* Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.cpp:
(WebKit::SourceBufferPrivateRemote::addTrackBuffer):
(WebKit::SourceBufferPrivateRemote::updateTrackIds):
(WebKit::SourceBufferPrivateRemote::bufferedSamplesForTrackId):
(WebKit::SourceBufferPrivateRemote::enqueuedSamplesForTrackID):
(WebKit::SourceBufferPrivateRemote::sourceBufferPrivateDidReceiveInitializationSegment):
* Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.h:
* Source/WebKit/WebProcess/GPU/media/TextTrackPrivateRemote.cpp:
(WebKit::TextTrackPrivateRemote::TextTrackPrivateRemote):
(WebKit::TextTrackPrivateRemote::setMode):
* Source/WebKit/WebProcess/GPU/media/TextTrackPrivateRemote.h:
* Source/WebKit/WebProcess/GPU/media/VideoTrackPrivateRemote.cpp:
(WebKit::VideoTrackPrivateRemote::VideoTrackPrivateRemote):
(WebKit::VideoTrackPrivateRemote::setSelected):
* Source/WebKit/WebProcess/GPU/media/VideoTrackPrivateRemote.h:
(WebKit::VideoTrackPrivateRemote::create):

Canonical link: https://commits.webkit.org/271599@main