[webkit-changes] [WebKit/WebKit] e0ce48: Safari is not using the intended AudioSession

[Dean Jackson]

  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: e0ce48fe5c2e1b20a22f817e081face321c7c541
      https://github.com/WebKit/WebKit/commit/e0ce48fe5c2e1b20a22f817e081face321c7c541
  Author: Dean Jackson <dino at apple.com>
  Date:   2023-04-20 (Thu, 20 Apr 2023)

  Changed paths:
    M LayoutTests/media/audio-session-category-at-most-recent-playback-expected.txt
    M LayoutTests/media/audio-session-category-at-most-recent-playback.html
    M LayoutTests/media/audio-session-category-expected.txt
    M LayoutTests/media/audio-session-category.html
    A LayoutTests/media/video-audio-session-mode-expected.txt
    A LayoutTests/media/video-audio-session-mode.html
    M Source/WebCore/PAL/pal/cocoa/AVFoundationSoftLink.h
    M Source/WebCore/PAL/pal/cocoa/AVFoundationSoftLink.mm
    M Source/WebCore/html/HTMLMediaElement.cpp
    M Source/WebCore/html/HTMLMediaElement.h
    M Source/WebCore/platform/audio/AudioSession.cpp
    M Source/WebCore/platform/audio/AudioSession.h
    M Source/WebCore/platform/audio/cocoa/MediaSessionManagerCocoa.mm
    M Source/WebCore/platform/audio/ios/AudioSessionIOS.h
    M Source/WebCore/platform/audio/ios/AudioSessionIOS.mm
    M Source/WebCore/platform/audio/mac/AudioSessionMac.h
    M Source/WebCore/platform/audio/mac/AudioSessionMac.mm
    M Source/WebCore/platform/mock/MockRealtimeAudioSource.cpp
    M Source/WebCore/testing/Internals.cpp
    M Source/WebCore/testing/Internals.h
    M Source/WebCore/testing/Internals.idl
    M Source/WebKit/GPUProcess/media/RemoteAudioSessionProxy.cpp
    M Source/WebKit/GPUProcess/media/RemoteAudioSessionProxy.h
    M Source/WebKit/GPUProcess/media/RemoteAudioSessionProxy.messages.in
    M Source/WebKit/GPUProcess/media/RemoteAudioSessionProxyManager.cpp
    M Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in
    M Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp
    M Source/WebKit/WebProcess/GPU/media/RemoteAudioSession.cpp
    M Source/WebKit/WebProcess/GPU/media/RemoteAudioSession.h
    M Source/WebKit/WebProcess/Speech/SpeechRecognitionRealtimeMediaSourceManager.cpp

  Log Message:
  -----------
  Safari is not using the intended AudioSession
https://bugs.webkit.org/show_bug.cgi?id=255681
rdar://107095456

Reviewed by Jer Noble.

WebKit never sets the AudioSession mode to anything other than "default", which
is not appropriate for video playback, especially in a spatial audio environment.
Fix this by exposing a way for the Media system to select an AudioSession mode,
and set it to MoviePlayback when needed.

* LayoutTests/media/audio-session-category-at-most-recent-playback-expected.txt:
* LayoutTests/media/audio-session-category-at-most-recent-playback.html:
* LayoutTests/media/audio-session-category-expected.txt:
* LayoutTests/media/audio-session-category.html:

* Source/WebCore/PAL/pal/cocoa/AVFoundationSoftLink.h: Expose the new mode constant.
* Source/WebCore/PAL/pal/cocoa/AVFoundationSoftLink.mm:

* Source/WebCore/html/HTMLMediaElement.cpp: Save the mode so that tests can query it.
(WebCore::HTMLMediaElement::playPlayer):
* Source/WebCore/html/HTMLMediaElement.h:
(WebCore::HTMLMediaElement::modeAtMostRecentPlayback const):

* Source/WebCore/platform/audio/AudioSession.h: Add a new type AudioSession::Mode that can be
set alongside the Category value.
(WTF::LogArgument<WebCore::AudioSession::Mode>::toString):
* Source/WebCore/platform/audio/AudioSession.cpp:
(WebCore::AudioSession::setCategory):
(WebCore::AudioSession::setCategoryOverride):
(WebCore::AudioSession::mode const):
(WebCore::convertEnumerationToString):

* Source/WebCore/platform/audio/cocoa/MediaSessionManagerCocoa.mm:
(WebCore::MediaSessionManagerCocoa::updateSessionState): Set the mode to VideoPlayback here
if `hasAudibleAudioOrVideoMediaType` is true.

* Source/WebCore/platform/audio/ios/AudioSessionIOS.h:
* Source/WebCore/platform/audio/ios/AudioSessionIOS.mm:
(WebCore::AudioSessionIOS::setCategory): Actually send the mode down to the AVAudioSession.
(WebCore::AudioSessionIOS::mode const):

* Source/WebCore/platform/audio/mac/AudioSessionMac.h:
* Source/WebCore/platform/audio/mac/AudioSessionMac.mm:
(WebCore::AudioSessionMac::setCategory): Accept the parameter, but we don't set anything on Mac.

* Source/WebCore/platform/mock/MockRealtimeAudioSource.cpp:
(WebCore::MockRealtimeAudioSource::startProducingData): Update call to setCategory.

* Source/WebCore/testing/Internals.cpp: Expose a way to test the mode.
(WebCore::Internals::audioSessionMode const):
(WebCore::Internals::modeAtMostRecentPlayback const):
* Source/WebCore/testing/Internals.h:
* Source/WebCore/testing/Internals.idl:

* Source/WebKit/GPUProcess/media/RemoteAudioSessionProxy.cpp:
(WebKit::RemoteAudioSessionProxy::setCategory):
* Source/WebKit/GPUProcess/media/RemoteAudioSessionProxy.h:
(WebKit::RemoteAudioSessionProxy::mode const):
* Source/WebKit/GPUProcess/media/RemoteAudioSessionProxy.messages.in:
* Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in:
* Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp:
* Source/WebKit/WebProcess/GPU/media/RemoteAudioSession.cpp:
(WebKit::RemoteAudioSession::setCategory):
(WebKit::RemoteAudioSession::mode const):
* Source/WebKit/WebProcess/GPU/media/RemoteAudioSession.h:
* Source/WebKit/WebProcess/Speech/SpeechRecognitionRealtimeMediaSourceManager.cpp:
    Update the call to setCategory to accept Mode.

* Source/WebKit/GPUProcess/media/RemoteAudioSessionProxyManager.cpp:
(WebKit::RemoteAudioSessionProxyManager::updateCategory):
    Iterate through the proxies to test if mode has been set.

Canonical link: https://commits.webkit.org/263201@main