[Webkit-unassigned] [Bug 221754] New: iOS 14: HLS live stream playback is getting extremely jerky permanently while microphone capture is enabled via getUserMedia

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=221754

            Bug ID: 221754
           Summary: iOS 14: HLS live stream playback is getting extremely
                    jerky permanently while microphone capture is enabled
                    via getUserMedia
           Product: WebKit
           Version: Safari 14
          Hardware: iPhone / iPad
                OS: iOS 14
            Status: NEW
          Severity: Critical
          Priority: P2
         Component: Media
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: up at nanocosmos.de

HLS live stream playback is getting extremely jerky permanently while microphone capture is enabled via getUserMedia. 
It is known that calling getUserMedia can cause short interruptions of the media playback. 
This is not the subject of this report. 

Severity 
The issue is business critical for us as it's disabling related use cases. 

Use case 
Audio-only or audio plus video webrtc upstream while watching a one-to-many HLS live stream 

Scope of the issue 

iOS versions 
The issue is related to iOS 14/Safari 14 and has not been recreated or reported for iOS 13 or earlier. 
We have recreated the problem on iOS 14.3, 14.4 and 14.5 public beta (18E5140k). 
We have received a report for iOS 14.2. We have no reliable knowledge about earlier iOS 14 versions atm. 

iOS devices 
The problem is reliably recreatable on: 
* iPhone 12 
* iPhone 11 
* iPhone XR 
The problem is not recreatable on: 
* iPhone X
* iPhone SE 2020

Audio devices 
The problem is recreatable with: 
* The internal microphone and speaker 
* Airpods pro headset with iPhone 12 in one setup 
* A JBL bluetooth headset with iPhone 12 in one setup 

The problem has not been recreatable with: 
* Wired earpods including microphone in all setups 
* External bluetooth speakers in all tests 
* Airpods pro headset with iPhone 11 in one setup 
* Airpods (external test, less reliable) 

Web page workflow 

One VideoElement is playing the HLS live stream unmuted. 
Tested codecs: H.264/AAC
Tested multiplex: fMP4 and ts 
Tested stream sources: Own service and youtube live 
Apple LL-HLS: no 

Audio capture is being enabled via getUserMedia. 
Including/excluding camera capture had no effect. 
Problem does not occur with camera-only capture. 
Tested several audio constraints without effect including: 
* Echo cancellation
* Auto gain
* Noise suppression
Adding/removing a MediaElement to preview the user media stream has no effect. 

The order of playback and capture start had no effect. 
Our use case is to start the playback first. 
No actual webrtc streams required to reproduce. 
Apart from the HLS playback there is no additional MediaElements or WebAudio involved. 

Description of the appearance 

As soon as the audio capture is getting active, video and audio of the 
media playback is getting very jerky, sometimes stalled, sometimes with 
very low update rates of 1-2 fps. Audio is often completely inaudible, 
sometimes very short bursts of audio occur. In two tests the playback recovered 
after around 5 minutes, in other the issue has been persistent. 

Disabling and enabling the capture session in the browser address bar is 
reliably recovering and reintroducing the issue. 

Sometimes the media playback can be recovered by applying pause/play, 
but this seems to be very unreliable. Furthermore the issue can reoccur 
either without interaction or forced by e.g. 
* Changing the mute state of the playback, even from unmuted to muted 
* Entering or exiting fullscreen mode
* Sometimes by tapping, scrolling or zooming the page 

How to recreate 
* Test page requires a getUserMedia call with audio:true and a VideoElement with live HLS source 
* Start video playback unmuted 
* Activate audio capture

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20210211/cc723db0/attachment-0001.htm>