[Webkit-unassigned] [Bug 266922] New: MediaElementAudioSourceNode doesn't pass audio sample data to subsequent node if based on MSE

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=266922

            Bug ID: 266922
           Summary: MediaElementAudioSourceNode doesn't pass audio sample
                    data to subsequent node if based on MSE
           Product: WebKit
           Version: Safari 17
          Hardware: All
                OS: macOS 14
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Web Audio
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: goncharov at wmspanel.com
                CC: cdumez at apple.com

MediaElementAudioSourceNode doesn't pass audio sample data to subsequent node in audio graph if video/audio element represented by the MediaElementAudioSourceNode has MediaSource object as a source. 
If however the video/audio element has Blob object with the same data as a source, then MediaElementAudioSourceNode passes audio data correctly.

There are two sample page links in the end of this description, that show buggy and correct behavior respectively. The problem is reproduced with AudioWorklet node, Analyser node and ScriptProcessor node (deprecated), so I used AudioWorklet node as the most relevant example.

Steps to reproduce:
1. Create MediaSource object:
    var mediaSource = new MediaSource;

2. Create object URL from mediaSource and assign it as media element's source:
    videoElement.src = URL.createObjectURL(mediaSource);

3. Wait for the mediaSource object to switch to 'open' state, add source buffer to it and then append AAC init segment and audio samples in ISO BMFF container.

4. On a user's gesture, instantiate AudioContext and create MediaElementAudioSourceNode interface:
    audioContext.createMediaElementSource(videoElement);

5. Assemble audio graph, connecting AudioWorklet node and destination node consequently. AudioWorklet processor has basic configuration that only transmits sample data values to the main thread in its process() method.

6. Observe sample values returned by AudioWorklet node processor.

Expected behavior: actual sample values in [-1, 1] range.
Actual behavior: all sample values are zeroes.

If the same audio samples are fed to the media element with Blob object, e.g.
    var blob = await fetchedData.blob();
    videoElement.src = URL.createObjectURL(blob);

then AudioWorklet node processor returns expected sample values.

Here are simple demo pages for both scenarios (please wait several seconds for AAC samples to download):

MediaSource related bug: https://dev.wmspanel.com/www_tmp/safari/safari_bug.html
Correct behavior with Blob source: https://dev.wmspanel.com/www_tmp/safari/safari_ok.html

Chrome and Firefox browsers show expected behavior on MacOS, Linux and Windows. The same issue is reproduced on iPadOS with all browsers.

It seems like this issue is connected to this one: https://bugs.webkit.org/show_bug.cgi?id=180696
I'm not sure that they have the same cause, but might overlap though.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20231228/20716265/attachment-0001.htm>