[webkit-changes] [WebKit/WebKit] 154de6: [iOS] voice activity detection is not working

youennf noreply at github.com
Mon Sep 23 10:08:02 PDT 2024


  Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: 154de6967e0b4ec901fe8e2fe6843622664e6c45
      https://github.com/WebKit/WebKit/commit/154de6967e0b4ec901fe8e2fe6843622664e6c45
  Author: Youenn Fablet <youenn at apple.com>
  Date:   2024-09-23 (Mon, 23 Sep 2024)

  Changed paths:
    M Source/WebCore/platform/mediastream/mac/BaseAudioSharedUnit.h
    M Source/WebCore/platform/mediastream/mac/CoreAudioSharedInternalUnit.h
    M Source/WebCore/platform/mediastream/mac/CoreAudioSharedUnit.cpp
    M Source/WebCore/platform/mediastream/mac/CoreAudioSharedUnit.h
    M Source/WebCore/platform/mediastream/mac/CoreAudioSharedUnit.mm
    M Source/WebCore/platform/mediastream/mac/MockAudioSharedUnit.mm

  Log Message:
  -----------
  [iOS] voice activity detection is not working
rdar://135942656
https://bugs.webkit.org/show_bug.cgi?id=280056

Reviewed by Eric Carlson.

On iOS, we first need to mute the VPIO unit and then set the voice activity listener.
We rework the listener enablement/disablement code so that it works for both macOS and iOS.

We introduce CoreAudioSharedUnit::shouldEnableVoiceActivityDetection which tells whether voice activity listener should be on.
On iOS, it will return true if VPIO is started, muted and the voice activity callback is set.
On macOS, we do not check whether VPIO is muted as this is not necessary.
This allows tabs that have been muted due to another page capturing to receive notifications of voice activity if the tab is visible.

We introduce CoreAudioSharedUnit::updateVoiceActiveDetection which will call CoreAudioSharedInternalUnit::setVoiceActivityDetection according CoreAudioSharedUnit::shouldEnableVoiceActivityDetection.
CoreAudioSharedUnit::updateVoiceActiveDetection is called from various places when some capture states change.

Tested on macOS and iOS devices.

* Source/WebCore/platform/mediastream/mac/BaseAudioSharedUnit.h:
(WebCore::BaseAudioSharedUnit::hasVoiceActivityListenerCallback const):
* Source/WebCore/platform/mediastream/mac/CoreAudioSharedInternalUnit.h:
* Source/WebCore/platform/mediastream/mac/CoreAudioSharedUnit.cpp:
(WebCore::CoreAudioSharedUnit::~CoreAudioSharedUnit):
(WebCore::CoreAudioSharedUnit::cleanupAudioUnit):
(WebCore::CoreAudioSharedUnit::startInternal):
(WebCore::CoreAudioSharedUnit::isProducingMicrophoneSamplesChanged):
(WebCore::CoreAudioSharedUnit::stopInternal):
* Source/WebCore/platform/mediastream/mac/CoreAudioSharedUnit.h:
* Source/WebCore/platform/mediastream/mac/CoreAudioSharedUnit.mm:
(WebCore::CoreAudioSharedUnit::shouldEnableVoiceActivityDetection const):
(WebCore::CoreAudioSharedUnit::updateVoiceActiveDetection):
(WebCore::CoreAudioSharedInternalUnit::setVoiceActivityDetection):
(WebCore::CoreAudioSharedUnit::enableMutedSpeechActivityEventListener):
(WebCore::CoreAudioSharedUnit::disableMutedSpeechActivityEventListener):
* Source/WebCore/platform/mediastream/mac/MockAudioSharedUnit.mm:
(WebCore::MockAudioSharedInternalUnit::setVoiceActivityDetection):

Canonical link: https://commits.webkit.org/284084@main



To unsubscribe from these emails, change your notification settings at https://github.com/WebKit/WebKit/settings/notifications


More information about the webkit-changes mailing list