[Webkit-unassigned] [Bug 218762] Stuttering audio from WebRTC in iOS 14.2

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=218762

--- Comment #17 from Teodor <teodor.atroshenko at gmail.com> ---
Still no workaround, but here is a short progress report on what we've tried:

- Trying to mix in white noise as dither through WebAudio API. However, they did not mix as expected in MediaStreamAudioDestinationNode. Furthermore, several times the white noise itself would stutter before any media arrived, but I assume this is purely due to CPU starvation when setting up the PeerConnection, and not related to this issue.

- Trying to delay the playback via DelayNode through WebAudio API. Regardless of the delay value, and whenever the value is increased, reduced or kept constant, the stuttering occurs whenever the stream is played back.

- Trying to forcefully restart offset computation by sending and/or receiving large DataChannel payloads at the beginning of the session. It didn't have an effect, and since audio is higher priority I doubt this can have an effect without SDP munging. Maybe something worth exploring.

- Trying to pause/mute/reattach the stream and trying to switch between devices at sender. Stuttering still occurs for "new" streams.

- Anything obvious like bitrate, codec and codec parameter manipulations. I guess dropping bitrate to zero would work, but picking it back up would likely trigger the same issue. Maybe worth experimenting.


What was not tried by us yet:

- Implementing "userland" buffer for audio frames and trying to counteract what the WebKit code is doing (skipping chunks of buffer) by repeating the frames at strategic offsets to force the continuous playback. Big problem with this approach, assuming it is even doable, is knowing when to stop - at some point the stuttering may end by itself (after reaching a stable buffer state) and we no longer want to be outputting duplicate frames any more.

- Attempting to capture video and audio separately, which may (or may not) allow us to forcefully desync audio and video tracks, allowing audio to catch up on its own. I am not sure this would work without also doing some SDP munging to deassociate video media lines from audio media lines. Another possibility is trying separate PeerConnections for video and audio streams.


Overall, based on the stats that we've captured, the issue only occurs if jitter goes above ~16ms (on our test bench), then once it goes down below ~12ms the stuttering disappears. This, of course, assumes that jitter at rest is at most around 10ms, otherwise it is theoretically possible to reach the edge of the buffer window again and get the stuttering back.


Skipping relay candidates, if you can, may help. At least for "nearby" connections, and ignoring other variable, as this largely depends on relay performance and whenever it is itself responsible for any jitter (for example if the relay does protocol switching between UDP and TCP). Stuttering is still there, but it stabilizes faster for us, which is better than never.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20201113/8e12fe7b/attachment-0001.htm>