[webkit-dev] Request for position on AudioContext.outputLatency

Wed Dec 15 09:16:49 PST 2021

(Copying Hongchan's reply to the list)

On Wed, Dec 15, 2021 at 5:44 PM Hongchan Choi <hongchan at chromium.org> wrote:

> Hi Youenn,
>
> A first good step would be to add the corresponding fingerprinting markup
>> to the web audio spec.
>>
>
> Do you believe the current language is not enough? Then please file an
> issue against the spec, so we can reopen the discussion in WG:
> https://github.com/WebAudio/web-audio-api/issues
>
> https://w3c.github.io/mediacapture-output/ allows listing devices that a
>> user granted.
>> In case devices are exposed to JS, exposing latency for those devices
>> seems fine.
>>
>
> Doesn't it require a full permission from the user? I am not sure whether
> exposing the latency value has the same severity as allowing the microphone
> access.
>
> I also wonder whether output latency might be useful outside of
>> AudioContext and whether it would make sense to expose this information on
>> MediaDevices as well. Could that be a device selection choice?
>>
>
> MediaTrackConstraints have "latency" field (
> https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackConstraints/latency),
> but I am not sure that is within the scope of the new API in question.
>
> The main issue is with the default system audio output. Passive filtering
>> should not be allowed in that case. For instance, if the AudioContext is
>> not producing audio, exposing latency does not make sense.
>>
>
> When AudioContext is suspended or not allowed to play, the latency should
> return zero value. This makes sense because the delay information from the
> audio infra changes every audio callback, but the suspended AudioContext
> won't have any calls from the device. Hence, the zero value.
>
> Can you elaborate what you meant by "passive filtering"?
>
> I am a bit unclear about when latency can change, for instance say audio
>> context is running and the default system speaker is changed. If it can
>> change, should there be some form of events to let the application know. Or
>> should it be left to things like MediaDevices.ondevicechange?
>>
>
> MediaDevices.ondevicechange is designed to serve the purpose you
> mentioned. Upon this event, the user can enumerate the device listing again
> to get the latest information - which is gated by the user permission.
>
> Cheers,
> Hongchan
>
> PS: Whom should I talk to about the WebKit mailing list subscription? When
> I log in at https://lists.webkit.org/mailman/listinfo/webkit-dev, I am
> still getting 403.
>
>
> On Wed, Dec 15, 2021 at 8:06 AM youenn fablet <youennf at gmail.com> wrote:
>
>> Thanks Hongchan,
>>
>> It is good to hear that these discussions are happening.
>> A first good step would be to add the corresponding fingerprinting markup
>> to the web audio spec.
>>
>> A few additional thoughts:
>>
>> https://w3c.github.io/mediacapture-output/ allows listing devices that a
>> user granted.
>> In case devices are exposed to JS, exposing latency for those devices
>> seems fine.
>> I also wonder whether output latency might be useful outside of
>> AudioContext and whether it would make sense to expose this information on
>> MediaDevices as well. Could that be a device selection choice?
>>
>> The main issue is with the default system audio output. Passive filtering
>> should not be allowed in that case. For instance, if the AudioContext is
>> not producing audio, exposing latency does not make sense.
>>
>> I am a bit unclear about when latency can change, for instance say audio
>> context is running and the default system speaker is changed. If it can
>> change, should there be some form of events to let the application know. Or
>> should it be left to things like MediaDevices.ondevicechange?
>>
>>
>> Le lun. 13 déc. 2021 à 20:00, Hongchan Choi <hongchan at chromium.org> a
>> écrit :
>>
>>> Hi Youenn,
>>>
>>> Thanks for your response.
>>>
>>> Say I switch from builtin speakers to Bluetooth headset using MacOS
>>>> system menu.
>>>>
>>>
>>> If changing a device doesn't reflect the current status correctly, the
>>> feature is not useful. That said, the accuracy/precision of a value being
>>> served is up to the user agent's discretion.
>>>
>>> If so, the spec should identify this as potential fingerprinting and
>>>> should provide mitigations.
>>>> And we should evaluate fingerprinting-free alternatives.
>>>
>>>
>>> We are aware of the issue and it is being discussed in the blink-dev I2S
>>> thread.
>>>
>>> What does PING WG think about this?
>>>
>>>
>>> Please note that Web Audio API is currently a W3C Recommendation.
>>> - The WG went through several PING reviews in 2020, for example:
>>> https://github.com/WebAudio/web-audio-api/issues/2061
>>> - One of device-related PING threads:
>>> https://github.com/w3cping/tracking-issues/issues/50
>>>
>>> As a result of the privacy discussion, the spec has a dedicated section
>>> on security and privacy:
>>> https://webaudio.github.io/web-audio-api/#priv-sec
>>>
>>> The relevant excerpt:
>>> "Fingerprinting via latency is also possible; it might be possible to
>>> deduce this from baseLatency and outputLatency. Mitigation strategies
>>> include adding jitter (dithering) and quantization so that the exact skew
>>> is incorrectly reported. Note however that most audio systems aim for low
>>> latency, to synchronise the audio generated by WebAudio to other audio or
>>> video sources or to visual cues (for example in a game, or an audio
>>> recording or music making environment). Excessive latency decreases
>>> usability and may be an accessibility issue."
>>>
>>> Cheers,
>>> Hongchan
>>>
>>>
>>> On Mon, Dec 13, 2021 at 1:15 AM youenn fablet <youennf at gmail.com> wrote:
>>>
>>>> Looking at
>>>> https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency, it
>>>> states that:
>>>> > If the audio output device is changed the outputLatency
>>>> <https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency> attribute
>>>> value will be updated accordingly.
>>>>
>>>> The use case seems ok, but I worry about fingerprinting if it means
>>>> this allows a web page to passively identify user speakers.
>>>> Say I switch from builtin speakers to Bluetooth headset using MacOS
>>>> system menu.
>>>>
>>>> If so, the spec should identify this as potential fingerprinting and
>>>> should provide mitigations.
>>>> And we should evaluate fingerprinting-free alternatives.
>>>> What does PING WG think about this?
>>>>
>>>> Le lun. 13 déc. 2021 à 09:39, Yoav Weiss via webkit-dev <
>>>> webkit-dev at lists.webkit.org> a écrit :
>>>>
>>>>> (Sent on behalf of Hongchan Choi, who failed to subscribe to this
>>>>> mailing list)
>>>>>
>>>>> Hey folks!
>>>>>
>>>>> AudioContext.outputLatency is to inform the time at which the first
>>>>> sample in the buffer is actually processed by the audio output device. This
>>>>> is useful when synchronizing the audio generated by Web Audio to other
>>>>> audio or video sources or to visual cues [1].
>>>>>
>>>>> This is already implemented in FireFox and we're looking to ship it in
>>>>> Chrome soon [2][3]. Would you all be interested in this feature?
>>>>>
>>>>> Thanks,
>>>>> Hongchan
>>>>>
>>>>> [1] https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency
>>>>> [2] https://bugzilla.mozilla.org/show_bug.cgi?id=1324552
>>>>> [3]
>>>>> https://groups.google.com/a/chromium.org/g/blink-dev/c/dTQniJNVVMY/m/hPFwY1fbBQAJ
>>>>>
>>>>> _______________________________________________
>>>>> webkit-dev mailing list
>>>>> webkit-dev at lists.webkit.org
>>>>> https://lists.webkit.org/mailman/listinfo/webkit-dev
>>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20211215/f49e5f21/attachment.htm>