[Webkit-unassigned] [Bug 106658] New: Web Audio Api CreateBuffer and decodeAudioData DOM exception 12 proposed solution?

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=106658

           Summary: Web Audio Api CreateBuffer and decodeAudioData DOM
                    exception 12 proposed solution?
           Product: WebKit
           Version: 528+ (Nightly build)
          Platform: All
        OS/Version: All
            Status: UNCONFIRMED
          Severity: Normal
          Priority: P2
         Component: JavaScriptCore
        AssignedTo: webkit-unassigned at lists.webkit.org
        ReportedBy: neuralll at gmail.com


Lot of people have trouble convincing Web Audio Api to play their files and are receiving weird DOM exception 12.

tested:
mobile safari 6.0.1
latest win7 chrome 24.0.1312.52m

reproduction steps: 

prepend  for example 100  bytes in front of any mp3.
observe DOM exception 12. on xhr/createBuffer/decodeAudioData attempt to decode or play.



I am aware that this is new and evolving technology and  I am very  thankful even for web audio api as it is now since its small miracle that happened to us.

both apis  miss  stream syncing on header boundary which is not used on pure stream files(i.e. no performance hit). But world is not perfect and there is bunch of data in streams these days. And mp3 or many aac/adts files are streaming file formats. streaming means that you can cut them anywhere or insert append anything (various tags even image artwork) decoder shouldnt care about unknown data. decoder should just seek until he finds header he knows and can decode.

I thrown together this temporary solution that seeks to nearest frame header start and passes data from this offset only.

mp3 or mp2 all start their header with 0XFFE and aac(adts) on oxFFF sync-word .therefore both will sync on 0xFFE. Yes its oversimplification
Here is the code I currently use to play previously not played files.

What I hate is that arrayBuffer doesnt have subarray() like its typed childs to return just different view from different offset instead of whole new array copy that slice() returns. if only webaudio api accepted typed arrays like Uint8Array alike as input but unfortunately the only way to create arrayBuffer back to keep type check happy seems huge slice() copy. thankfully usually only one or two seeks are needed. and frames are small i.e. no more than 300 or 600 bytes checked.
Still seeking or pre-decoding mp3 or aac in js doesn't seem right. So i have feeling that frame-sync should be part of api. After all any reasonable streaming format decoder starts with it. Yes there is media element source but that is disabled on iOS and i thing one shot sources shouldn't be picky about files. many people have no idea how to frame align their files in hex editor. probably  mentioning of mandatory frame alignment in documentation will help steer people. what you guys think?

// current temporary quick JS hack that plays files with tags or artwork or partial cuts too
// simply prepend bunch of bytes  in front of previously playable mpg/aac(adts) to simulate the  ones with embedded artwork or tags.

function syncStream(node){
    var buf8 = new Uint8Array(node.buf); 
    buf8.indexOf = Array.prototype.indexOf;
    var i=node.sync, b=buf8;
    while(1) {
        node.retry++;
        i=b.indexOf(0xFF,i); if(i==-1 || (b[i+1] & 0xE0 == 0xE0 )) break;
        i++;
    }
    if(i!=-1) {
        var tmp=node.buf.slice(i); //carefull there it returns copy
        delete(node.buf); node.buf=null;
        node.buf=tmp;
        node.sync=i;
        return true;
    }
    return false;
}

function decode(node) {
    try{
        context.decodeAudioData(node.buf,
        function(decoded){
            node.source  = context.createBufferSource();
            node.source.connect(context.destination);
            node.source.buffer=decoded; 
            node.source.noteOn(node.time);
        },
        function(){ 
            if(syncStream(node)) decode(node);
        });
    } catch(e) {
        log('decode exception',e.message);
    }
}

function playSound(node) { 
    node.xhr = new XMLHttpRequest();
    node.xhr.onload=function(){  
        node.buf=node.xhr.response;
        node.sync=0;
        node.retry=0;
        decode(node);
    }
    node.xhr.open("GET", node.url, true); 
    node.xhr.responseType = "arraybuffer"; 
    node.xhr.send();
}

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.