[Webkit-unassigned] [Bug 176282] autoplay hidden video element with MediaStream src that has audio track

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=176282

Dan <kentbriggs5 at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kentbriggs5 at gmail.com

--- Comment #20 from Dan <kentbriggs5 at gmail.com> ---
Just tested out the mediaStream approach @Youenn mentioned. I tried two different methods:
1. audioElement.srcObject = new MediaStream(allTheTracks) every time a new audio track came in.
2. Used the same mediaStream object the entire time and added tracks dynamically with mediaStrack.addTrack()

I got similar results for both methods. Sometimes, everything worked fine. Other times, only some tracks would play audio. I wasn't able to reliably reproduce the issue, it seemed to happen a random % of the time (about 50%). The times when audio wasn't playing, some of the tracks had enabled === false. Interestingly, the tracks that had enabled === false could often be heard and the tracks that had enabled === true couldn't. Very odd!

For the most part, everything works fine if I render a separate audio element for each audio track. The only weird thing I noticed here is that sometimes audio can't be heard even when all indications say it should be (audio track is enabled, audio element is playing, etc). Interestingly, this issue seemed to go away when adding the viewport meta tag to my html <meta name="viewport" content="width=device-width, initial-scale=1.0">. I'm wondering if this weird behaviour has something to do with the audio elements being rendered out of the viewport. I tried to reliably reproduce this issue but wasn't able to. It seemed to happen randomly when not including the meta tag. I'd refresh the page without changing the viewport and sometimes I could hear all the tracks, sometimes it couldn't. I don't seem to get the issue after adding that meta tag though.

Web Audio also worked for me. I didn't test it enough to know if there are lip-sync issues though as @Ben mentioned.

Just for reference, I ran these tests on an iPhone 5s and iPad Mini 2. I needed host candidates so I had the viewer grant device access with getUserMedia() then I stopped the tracks immediately before viewing remote streams from other devices.

I'm wondering if we can add a rule to allow multiple video elements to play audio if they are backed by a mediaStream. I agree with the comments above that this limitation is quite inconvenient for WebRTC. I imagine it will prevent some companies from supporting Safari in their WebRTC applications.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20190203/9dcbff59/attachment.html>