[webkit-changes] [WebKit/WebKit] d7e69a: [AVFObjC] [AVPlayerLayer videoPerformanceMetrics] ...

[Jean-Yves Avenard]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: d7e69a605f2ae52d006f4716edccd688fc6cba17
      https://github.com/WebKit/WebKit/commit/d7e69a605f2ae52d006f4716edccd688fc6cba17
  Author: Jean-Yves Avenard <jya at apple.com>
  Date:   2024-01-26 (Fri, 26 Jan 2024)

  Changed paths:
    M LayoutTests/media/media-source/media-source-video-playback-quality-expected.txt
    M LayoutTests/media/media-source/media-source-video-playback-quality.html
    M Source/WebCore/html/HTMLMediaElement.cpp
    M Source/WebCore/platform/PlatformMediaError.cpp
    M Source/WebCore/platform/PlatformMediaError.h
    M Source/WebCore/platform/graphics/MediaPlayer.cpp
    M Source/WebCore/platform/graphics/MediaPlayer.h
    M Source/WebCore/platform/graphics/MediaPlayerPrivate.cpp
    M Source/WebCore/platform/graphics/MediaPlayerPrivate.h
    M Source/WebCore/platform/graphics/VideoPlaybackQualityMetrics.h
    M Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.messages.in

  Log Message:
  -----------
  [AVFObjC] [AVPlayerLayer videoPerformanceMetrics] may deadlock
https://bugs.webkit.org/show_bug.cgi?id=268136
rdar://121637422

Reviewed by Youenn Fablet.

A call to [AVPlayerLayer videoPerformanceMetrics] may deadlock if the AVPlayer is
also waiting in the main thread for more data to be delivered.
This commonly occurs when there are lots of videos playing at once and result in
a blocked GPU process.

Dealing with getting the networking data being delivered on the main thread will be
dealt in bug 235353.
For now, call [AVPlayerLayer videoPerformanceMetrics] on a different WorkQueue.
The RemoteMediaPlayerProxy already regularly polls the MediaPlayerPrivate for the
videoPlaybackQualityMetrics, so performing the actions asynchronously doesn't have
a measurable impact.
And it does prevent the GPU process to be deadlock.

Fly-by: While the VideoPlaybackQualityMetrics were properly calculated
and updated in the GPU process at a set interval, the values weren't
being sent to the Content Process except under some limited circumstances
(video got paused, started playing or stall). As such, retrieving the
VideoPlaybackQuality from the video element would typically always return
the same values.

* LayoutTests/media/media-source/media-source-video-playback-quality-expected.txt:
* LayoutTests/media/media-source/media-source-video-playback-quality.html:
The quality metrics are calculated every 0.25s (250ms) but we were only
waiting for 50ms.
* Source/WebCore/html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::getVideoPlaybackQuality):
* Source/WebCore/platform/PlatformMediaError.cpp: Adding new error code.
A prior change was incomplete and failed to add the serialisation data.
(WebCore::convertEnumerationToString):
* Source/WebCore/platform/PlatformMediaError.h:
* Source/WebCore/platform/graphics/MediaPlayer.cpp:
(WebCore::MediaPlayer::asyncVideoPlaybackQualityMetrics):
* Source/WebCore/platform/graphics/MediaPlayer.h:
* Source/WebCore/platform/graphics/MediaPlayerPrivate.cpp:
(WebCore::MediaPlayerPrivateInterface::asyncVideoPlaybackQualityMetrics):
* Source/WebCore/platform/graphics/MediaPlayerPrivate.h:
* Source/WebCore/platform/graphics/VideoPlaybackQualityMetrics.h:
(WebCore::VideoPlaybackQualityMetrics::operator+=):
* Source/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h:
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
(WebCore::MediaPlayerPrivateAVFoundationObjC::asyncVideoPlaybackQualityMetrics):
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp:
(WebKit::RemoteMediaPlayerProxy::maybeUpdateCachedVideoMetrics):
(WebKit::RemoteMediaPlayerProxy::updateCachedVideoMetrics):
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h:
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
(WebKit::MediaPlayerPrivateRemote::updatePlaybackQualityMetrics):
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.messages.in:
Add specific message for sending the VideoPlaybackQualityMetrics, as sending
the entire cached state is expensive (particularly reading the video time).

Canonical link: https://commits.webkit.org/273580@main