[webkit-changes] [WebKit/WebKit] d9ce6b: REGRESSION(274311 at main) YouTube.com in safari Airp...

[Jean-Yves Avenard]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: d9ce6b4272d071babd1ff0ee646056cfb06f8b37
      https://github.com/WebKit/WebKit/commit/d9ce6b4272d071babd1ff0ee646056cfb06f8b37
  Author: Jean-Yves Avenard <jya at apple.com>
  Date:   2024-03-05 (Tue, 05 Mar 2024)

  Changed paths:
    M Source/WebCore/SourcesCocoa.txt
    M Source/WebCore/WebCore.xcodeproj/project.pbxproj
    M Source/WebCore/platform/graphics/MediaPlaybackTarget.h
    M Source/WebCore/platform/graphics/avfoundation/MediaPlaybackTargetCocoa.h
    M Source/WebCore/platform/graphics/avfoundation/MediaPlaybackTargetCocoa.mm
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlaybackTargetPickerMac.mm
    M Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm
    M Source/WebCore/platform/graphics/cocoa/MediaPlaybackTargetContext.h
    R Source/WebCore/platform/graphics/cocoa/MediaPlaybackTargetContext.mm
    R Source/WebCore/platform/graphics/cocoa/MediaPlaybackTargetContext.serialization.in
    M Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.h
    M Source/WebCore/platform/mock/MediaPlaybackTargetMock.cpp
    M Source/WebCore/platform/mock/MediaPlaybackTargetMock.h
    M Source/WebCore/platform/mock/MediaPlaybackTargetPickerMock.cpp
    M Source/WebKit/DerivedSources-input.xcfilelist
    M Source/WebKit/DerivedSources.make
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h
    M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in
    M Source/WebKit/GPUProcess/media/ios/RemoteMediaSessionHelperProxy.cpp
    M Source/WebKit/Platform/SourcesCocoa.txt
    A Source/WebKit/Platform/cocoa/MediaPlaybackTargetContextSerialized.h
    A Source/WebKit/Platform/cocoa/MediaPlaybackTargetContextSerialized.mm
    A Source/WebKit/Platform/cocoa/MediaPlaybackTargetContextSerialized.serialization.in
    M Source/WebKit/Scripts/webkit/messages.py
    M Source/WebKit/SourcesCocoa.txt
    M Source/WebKit/UIProcess/WebPageProxy.cpp
    M Source/WebKit/WebKit.xcodeproj/project.pbxproj
    M Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp
    M Source/WebKit/WebProcess/GPU/media/ios/RemoteMediaSessionHelper.cpp
    M Source/WebKit/WebProcess/GPU/media/ios/RemoteMediaSessionHelper.h
    M Source/WebKit/WebProcess/GPU/media/ios/RemoteMediaSessionHelper.messages.in
    M Source/WebKit/WebProcess/WebPage/WebPage.h
    M Source/WebKit/WebProcess/WebPage/WebPage.messages.in
    M Source/WebKit/WebProcess/WebPage/mac/WebPageMac.mm

  Log Message:
  -----------
  REGRESSION(274311 at main) YouTube.com in safari Airplay connects to ATV receiver, no Video plays but only Audio
https://bugs.webkit.org/show_bug.cgi?id=270437
rdar://123467550

Reviewed by Youenn Fablet.

When activating an AirPlay target on a currently playing media element,
the UI process was sending a MediaPlaybackTargetContext to the web process, which
would then pass it to the GPUP's AVPlayer.
Following 274311 at main, the AVOutputContext found in the MediaPlaybackTargetContext
would be serialized/deserialized at each IPC's endpoints. This relied on the
generic AVOutputContext IPC serialization introduced in 271664 at main.
However, re-constructing an AVOutputContext in the WP is not allowed by
the sandbox. An AVOutputContext's serialization is made of a global identification
string which can be looked up in a global table. This look-uo would always
fail in the WP resulting in an null AVOutputContext being generated.
Subsequently, the GPUP would receive this null AVOutputContext causing the
AirPlay command to never reach the GPUP.

On iOS devices similarly, the GPUP sends the notification that a new AVOutputContext had
been selected, sends it to the WP which then re-send it for the GPUP for processing.
As above, the transit through the WP would null the AVOutputContext.

We revert to the behaviour pre-274311 at main which would use a transient SerializedAVOutputContext
to pass a serialized AVOutputContext, with a cache of hasActiveRoute which would
only be reconstructed in the GPUP.
In order to keep the security aspect intended by 274311 at main we introduce two
new classes: MediaPlaybackTargetSerialized and MediaPlaybackTargetContextSerialized
which will wrap either a MediaPlaybackTargetContextCocoa or a MediaPlaybackTargetContextMock.
The underlying AVOutputContext is now never accessed in the WP. Instead we only pass
the underlying AVOutputContext's global ID.
We split MediaPlaybackTargetContext into three sub-classes:
- Cocoa/AVOutputContext
- Mock
- Serialized

The MediaPlaybackTargetSerialized now uses the new serialization framework.

Manually tested to work with an ATV and a macOS sonoma as AirPlay target.
Some work covered by MediaPlaybackTargetMock tests; but infrastructure doesn't
allow for comprehensive regression tests.

* Source/WebCore/SourcesCocoa.txt:
* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/platform/graphics/MediaPlaybackTarget.h:
* Source/WebCore/platform/graphics/avfoundation/MediaPlaybackTargetCocoa.h:
(isType):
* Source/WebCore/platform/graphics/avfoundation/MediaPlaybackTargetCocoa.mm:
(WebCore::MediaPlaybackTargetContextCocoa::MediaPlaybackTargetContextCocoa):
(WebCore::MediaPlaybackTargetContextCocoa::deviceName const):
(WebCore::MediaPlaybackTargetContextCocoa::hasActiveRoute const):
(WebCore::MediaPlaybackTargetContextCocoa::supportsRemoteVideoPlayback const):
(WebCore::MediaPlaybackTargetCocoa::create):
(WebCore::MediaPlaybackTargetCocoa::MediaPlaybackTargetCocoa):
(WebCore::MediaPlaybackTargetCocoa::~MediaPlaybackTargetCocoa): Deleted.
(WebCore::toMediaPlaybackTargetCocoa): Deleted.
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlaybackTargetPickerMac.mm:
(WebCore::MediaPlaybackTargetPickerMac::playbackTarget):
* Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
(WebCore::MediaPlayerPrivateAVFoundationObjC::setWirelessPlaybackTarget):
* Source/WebCore/platform/graphics/cocoa/MediaPlaybackTargetContext.h:
(WebCore::MediaPlaybackTargetContext::MediaPlaybackTargetContext):
(WebCore::MediaPlaybackTargetContext::mockState const): Deleted.
(WebCore::MediaPlaybackTargetContext::outputContext const): Deleted.
(WebCore::MediaPlaybackTargetContext::encodingRequiresPlatformData const): Deleted.
(): Deleted.
* Source/WebCore/platform/graphics/cocoa/MediaPlaybackTargetContext.mm: Removed.
* Source/WebCore/platform/mock/MediaPlaybackTargetMock.cpp:
(WebCore::MediaPlaybackTargetMock::create):
(WebCore::MediaPlaybackTargetMock::MediaPlaybackTargetMock):
(WebCore::toMediaPlaybackTargetMock): Deleted.
* Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.h: Add missing forward declaration.
* Source/WebCore/platform/mock/MediaPlaybackTargetMock.h:
(isType):
(WebCore::MediaPlaybackTargetMock::state const): Deleted.
* Source/WebCore/platform/mock/MediaPlaybackTargetPickerMock.cpp:
(WebCore::MediaPlaybackTargetPickerMock::playbackTarget):
* Source/WebKit/DerivedSources-input.xcfilelist:
* Source/WebKit/DerivedSources.make:
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp:
(WebKit::RemoteMediaPlayerProxy::setWirelessPlaybackTarget):
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h:
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
* Source/WebKit/GPUProcess/media/ios/RemoteMediaSessionHelperProxy.cpp:
(WebKit::RemoteMediaSessionHelperProxy::activeVideoRouteDidChange):
* Source/WebKit/Platform/SourcesCocoa.txt:
* Source/WebKit/Platform/cocoa/MediaPlaybackTargetContextSerialized.h: Added.
(isType):
* Source/WebKit/Platform/cocoa/MediaPlaybackTargetContextSerialized.mm: Added.
(WebKit::MediaPlaybackTargetContextSerialized::MediaPlaybackTargetContextSerialized):
(WebKit::MediaPlaybackTargetContextSerialized::platformContext const):
* Source/WebKit/Platform/cocoa/MediaPlaybackTargetContextSerialized.serialization.in: Renamed from Source/WebCore/platform/graphics/cocoa/MediaPlaybackTargetContext.serialization.in.
* Source/WebKit/Scripts/webkit/messages.py:
(conditions_for_header):
* Source/WebKit/SourcesCocoa.txt:
* Source/WebKit/UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::setMockMediaPlaybackTargetPickerState):
(WebKit::WebPageProxy::Internals::setPlaybackTarget):
* Source/WebKit/WebKit.xcodeproj/project.pbxproj:
* Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
(WebKit::MediaPlayerPrivateRemote::setWirelessPlaybackTarget):
* Source/WebKit/WebProcess/GPU/media/ios/RemoteMediaSessionHelper.cpp:
(WebKit::RemoteMediaSessionHelper::activeVideoRouteDidChange):
* Source/WebKit/WebProcess/GPU/media/ios/RemoteMediaSessionHelper.h:
* Source/WebKit/WebProcess/GPU/media/ios/RemoteMediaSessionHelper.messages.in:
* Source/WebKit/WebProcess/WebPage/WebPage.h:
* Source/WebKit/WebProcess/WebPage/WebPage.messages.in:
* Source/WebKit/WebProcess/WebPage/mac/WebPageMac.mm:
(WebKit::WebPage::playbackTargetSelected const):

Canonical link: https://commits.webkit.org/275730@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications