[Webkit-unassigned] [Bug 179411] getUserMedia constraints have no affect

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=179411

--- Comment #2 from Kerry Davis <kdavis at airbridgelabs.com> ---
Responding to : "getUserMedia with echoCancellation false should work on both MacOs and iOS. Can you verify whether this is working in latest iOS/MacOS Safari versions?"

Here is what I have found.

First, yes there is still a problem in that the current code I am using (which includes echoCancellation false) does not appear to turn echo cancellation off. MacOS for sure but since I really have nothing to compare it against on iOS (since everything runs on top of the iOS Safari webkit) I have nothing to compare it with but what results I have are at the end of this report.

Second, I have worked on/with many telecom standards bodies. It is my opinion and experience that allocation of a streaming resource of any kind should by default result in a filterless (aka raw) stream and never default with filters to the stream on. I have always assumed that this is because on bring up (another area i have quite a bit of embedded software experience in) you always want the hardware to come up in a known state for the software (even if, by some weird chance, the hardware actually enables the filter as it is a HW assisted filter...in which case the lowest level firmware should immediate disable all filters). I don't sit or have any real influence on the webRTC standard but I believe presenting a stream in a filterless initial/default state has real benefits in both software and HW debugging and operation (i.e. unforeseen cold resets)

Third, with respect to the second point above, evidently someone else (actually everybody else) either agrees or accidently agrees with allocating a audio stream raw by default. This is how Chrome, Opera, and Firefox are working now (as of nov 2018) on my macbook running MacOS.

How to reproduce: 

First: you need to get my airbridge communication extension for chrome from the chrome extension store. https://chrome.google.com/webstore/detail/airbridge/mblkfpphiadhncblckopllohojmjdoee

Second: AirBridge digital pulses will operate across 5 separate audio bands roughly placed at 3,8,14,18,20 KHz. Select the HIGH, or MEDIUM transmit frequency band by clicking on the extension icon (which should be green when the extension is active). Then do a google IMAGE search on any topic (cute puppies is my goto favorite). When you mouse over a puppy image in the search results you should now see my airbridge logo pop up and when you click the logo you should hear a click which is the default sound that provides feedback to the user that a pulse was sent (useful for the higher bands and us old people)


Now open up 4 instances of a test page I created containing my airbridge receiver code (https://www.airbridgelabs.com/s/sbc2.html?sd=5 .. this page should come up with the AirBridge receiver active...green star flashing on lower screen) in desktop browsers (Safari, Opera, Firefox and Chrome) all running on the same MacBook Air and the same code without conditionals.

The following code is used for getUserMedia()

navigator.mediaDevices.getUserMedia({audio: { 
            echoCancellation: false,
            noiseSuppression: false,
            autoGainControl: false,
            googEchoCancellation: false,
            googAutoGainControl: false,
            googExperimentalAutoGainControl: false,
            googNoiseSuppression: false,
            googExperimentalNoiseSuppression: false,
            googHighpassFilter: false,
            googTypingNoiseDetection: false,
            googBeamforming: false,
            googArrayGeometry: false,
            googAudioMirroring: false,
            googAudioMirroring: false,
            googNoiseReduction: false,
            mozNoiseSuppression: false,
            mozAutoGainControl: false
            }, 
        video: false})
    .then(handleABSuccess)
    .catch(function(err) {

        testlog (err.message+": Access to the microphone is necessary to receive AirBridge command pulses.\n\nHowever, microphone access was denied by the user (or the operating system) upon installation.\n\nTry resetting browser settings and redownloading this app or extension.");
        return (1);
    });

Now click on an image from the google search and look at each of the pages you just opened. All of them EXCEPT for Safari should now be displaying the image in a popup window. If not then tweak the volume up or down a bit but for me it usually works even down at one volume bar and all the way up to max volume (make sure you hear the confirmation beep/click and you can change to a couple other or no confirmation sound in the extension settings). If you don't hear the confirmation then refresh the search page and click a different image.

Why do I think this is a problem with audio filters not being disabled? Simply because if you repeat the same experiment from above except you change the aibridge transmit frequency to either 8K or LOW you will then see that the image starts also appearing in the Safari browser. This is because many audio filters, including and maybe especially echo cancellation and certainly compression algorithms (but compression is not the case here) will attenuate the upper frequencies but leave the lower ones (and my FFT analyser node) almost untouched.

In an additional test I actually removed the filters in my code and ran the above test again. EX.

navigator.mediaDevices.getUserMedia({audio: { 


            }, 
        video: false})
    .then(handleABSuccess)
    .catch(function(err) {

        testlog (err.message+": Access to the microphone is necessary to receive AirBridge command pulses.\n\nHowever, microphone access was denied by the user (or the operating system) upon installation.\n\nTry resetting browser settings and redownloading this app or extension.");
        return (1);
    });

What I found was that desktop Chrome && Opera both appear to still work at the high frequency bands while Safari only works at the LOW and 8K bands and Firefox only works at the LOW frequency bands (meaning its audio filters are more aggressive than Safari but at least I can turn them OFF). This makes sense if you consider that someone at both Chrome and Opera believe as I do that the default state of the newly created stream should have filters disabled, Firefox comes up with filters enabled (unless we explicitly disable them), and Safari has filters enabled whether you turn them off explicitly or not.

iPhone results.
Running iOS 12.1 on iPhone 7 I can't even fully load my code in Chrome, Firefox, Opera or Edge. I suspect this is because they all told me they don't support webRTC getUserMedia and my code does not handle that well so it gets stuck. As for Safari, my code does come up and everything appears to work but no pulses, even in the low bands, are received. In the debug I can see that the problem is simple that the analyser data is all 0's. This is exactly the same code that is working in the desktop Safari and I am getting no errors so I have no idea why it is not working in mobile Safari yet works in desktop version???

And interestingly, when I originally wrote this bug, I remember that mobile chrome on Android did NOT default to filters disabled as is the case in desktop Chrome. So much for object inheritance and cross compilers lol.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20181118/679bb005/attachment-0001.html>