[Webkit-unassigned] [Bug 224279] New: OfflineAudioContext startRendering fail after N calls

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=224279

            Bug ID: 224279
           Summary: OfflineAudioContext startRendering fail after N calls
           Product: WebKit
           Version: WebKit Nightly Build
          Hardware: All
                OS: All
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Web Audio
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: jasonlmcaffee at gmail.com
                CC: cdumez at apple.com

Created attachment 425392

  --> https://bugs.webkit.org/attachment.cgi?id=425392&action=review

iOS 14.4.2 errors after 26 calls to startRendering

Safari Desktop with latest Webkit Version 14.0.3 (16610.4.3.1.7) on Big Sur
Safari latest version for iOS 14.4.2
Safari on iOS 14.1

Hello,
I work on an application that allows users to slow down audio so they can play piano alongside of it.  
The performance of the pitchshifting done by SoundTouchJS (https://github.com/cutterbl/SoundTouchJS) degrades significantly with iOS 14.4 (confirmed on 14.4.1 and 14.4.2).  I believe iOS 14.4 has a performance degradation that affects the script processor node's onaudioprocess.
As a result, we are forced to abandon realtime processing, and opt instead to render slowed down audio with pitch correction via OfflineAudioContext (similar to HTML Audio's playRate functionality).  
We have a Single Page Application, and users may change the speed of the audio dozens of time for a given session.

We encountered a problem where the OfflineAudioContext creation starts to fail or cause the browser to crash after being called 24-60 times.

For desktop with Big Sur and latest Safari version 14.0.3, the page will either be reloaded or crash after ~60 calls to startRendering.

For iOS 14.4.2 and 14.1, calls to instantiate new OfflineAudioContext fail after ~26 calls with the error "The string did not match the expected pattern."
If you get that error ~5 times, and try calling context.createBuffer, you will receive the error "Channel was not able to be created".
If you refresh the page the error will immediately occur, and there does not seem to be a way to recover, other than to close the tab and open a new one.

I put together a codepen which demonstrates the issue:
https://codepen.io/jasonmcaffee/pen/GRrvLxX

```
const AudioContext = window.AudioContext || window.webkitAudioContext;
const OfflineAudioContext = window.OfflineAudioContext || window.webkitOfflineAudioContext;
const audioUrl = "https://playground-assets-sean.s3.amazonaws.com/r1/00/00/00/3c/slice_59_resource_178_20210308_13h53m55s_mst.mp3";

//demonstrate that iOS starts crashing or erroring when performing offline renders more than N times.
async function main(){
  const context = new AudioContext();
  const audioBuffer = await fetchAndDecodeAudioData({url: audioUrl, context});
  for(let i = 0; i < 60; ++i){
    print(`rendering ${i}`);
    try{
      const result = await renderOffline(audioBuffer); 
    }catch(e){
      print(`error rendering offline: `, e.message);
      createBuffers();
    }
  }
  print(`done rendering`);
}

//demonstrate that after receiving the error from the OfflineAudioContext creation, creation of audio buffers fail.
async function createBuffers(){
  const context = new AudioContext();
  print(`creating buffers`);
  for(let i = 0; i < 1; ++i){
    try{
      const buffer = context.createBuffer(2, 44100 * 10, 44100);  
    }catch(e){
      print(`error creating buffer: `, e.message);
    }
  }
}

async function renderOffline(audioBuffer: AudioBuffer){
  let offlineAudioContext = new OfflineAudioContext(audioBuffer.numberOfChannels, audioBuffer.duration * audioBuffer.sampleRate, audioBuffer.sampleRate);
  let source = offlineAudioContext.createBufferSource();
  source.buffer = audioBuffer;
  source.connect(offlineAudioContext.destination);
  return new Promise((resolve, reject)=>{
    offlineAudioContext.oncomplete = (e)=>{
      let renderedBuffer = e.renderedBuffer;
      resolve(renderedBuffer);
      //attempt to cleanup in hopes it will resolve the issue (Narrator: it doesnt)
      source.buffer = null;
      cleanupSource(source);
      offlineAudioContext.oncomplete = undefined;
      offlineAudioContext = undefined;
      source = undefined;
      renderedBuffer = undefined;
    };
    source.start();
    offlineAudioContext.startRendering();//ios doesn't support promise version
  });

}

async function fetchAndDecodeAudioData({url, context}: {url: string; context: AudioContext}): Promise<AudioBuffer> {
  const response = await fetch(url, {cache: "force-cache"});
  const arrayBuffer = await response.arrayBuffer();
  return decodeAudioData(context, arrayBuffer);
}

async function decodeAudioData(context: AudioContext, arrayBuffer: ArrayBuffer): Promise<AudioBuffer>{
  return new Promise((resolve, reject) => context.decodeAudioData(arrayBuffer, resolve, reject));
}

//attempt to use a stracth buffer to force safari to release references to buffers (doesnt work for this issue)
const context = new AudioContext();
let scratchBuffer = context.createBuffer(2,1,44100);
function getScratchBuffer(){
  return scratchBuffer;
}

function cleanupSource(source: AudioBufferSourceNode | undefined){
  if(!source){ return; }
  try{
    source.stop();
  }catch(e){}
  source.onended = null;
  source.disconnect(0);
  try{source.buffer = getScratchBuffer();}catch(e){}
}

const outputEl = document.querySelector('#output');
function print<T extends unknown>(...params: T[]){
  const message = params.map(p => JSON.stringify(p).replace(/['"]+/g, '')).join(` `);
  outputEl.innerHTML += `${message}<br/>`;
}

main();


```

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20210407/87986575/attachment-0001.htm>