[Webkit-unassigned] [Bug 264894] New: VideoEncoder produces no frames with latencyMode "realtime" when framerate/bitrate are not given

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=264894

            Bug ID: 264894
           Summary: VideoEncoder produces no frames with latencyMode
                    "realtime" when framerate/bitrate are not given
           Product: WebKit
           Version: Safari Technology Preview
          Hardware: Mac (Apple Silicon)
                OS: macOS 14
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Media
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: jacob at bandes-stor.ch

VideoEncoder with `latencyMode: "realtime"` but no `bitrate` or `framerate` specified does not output any frames, nor error messages.

Related: https://bugs.webkit.org/show_bug.cgi?id=264893 & https://bugs.webkit.org/show_bug.cgi?id=258669

Steps to reproduce:
1. Visit https://codesandbox.io/s/cocky-ioana-pjyz5v?file=/src/index.mjs
2. Open the console tab and observe the output is "{keyFrames: 0, deltaFrames: 0}" -- i.e. no frames were output whatsoever. No error messages are present.
3. Uncomment `framerate: fps` and `bitrate: 100000` and observe that the output changes to "{keyFrames: 1, deltaFrames: 19}".

Expected behavior:
If encoding without these parameters is not supported, then the browser should not claim to support the configuration (via isConfigSupported and by not calling the error callback).


Full code below:

```
import "./styles.css";

async function main() {
  const width = 200;
  const height = 100;
  const img = new ImageData(width, height);

  for (let r=0;r<height;r++) {
    for (let c=0;c<width;c++) {
      img.data[(r*width+c)*4+0] = Math.floor(r/height*255);
      img.data[(r*width+c)*4+1] = Math.floor(c/width*255);
      img.data[(r*width+c)*4+2] = 127;
      img.data[(r*width+c)*4+3] = 255;
    }
  }

  const bitmap = await createImageBitmap(img);

  let keyFrames=0;
  let deltaFrames=0;
  const encoder = new VideoEncoder({
    output: (chunk, metadata) => {
      if (chunk.type==="key") {
        ++keyFrames;
      } else if (chunk.type === "delta") {
        ++deltaFrames;
      }
      console.log('output chunk',chunk,metadata);
    },
    error: (err) => {
      console.error('encode error', err)
    }
  })

  encoder.addEventListener("dequeue", (event) => {
    console.log('dequeued')
  })

  const fps = 30;

  encoder.configure({
    codec: "avc1.42001f", // Baseline profile (42 00) with level 3.1 (1f)
    width,
    height,
    latencyMode: "realtime",
    // framerate: fps,
    avc: { format: "annexb" },
    // hardwareAcceleration:"prefer-software"
    // bitrate:100000,
    // bitrate: 10000000,
  });

  for (let i=0; i < 20; i++) {
    const frame = new VideoFrame(bitmap, {
      timestamp:i * 1/fps * 1e6,
      duration:1/fps * 1e6,
    });
    encoder.encode(frame, {keyFrame: i === 0});
    frame.close();
  }

  bitmap.close();
  console.log("sent frames");

  await encoder.flush();
  console.log("flushed", {keyFrames,deltaFrames})
  encoder.close();
  console.log("closed", {keyFrames,deltaFrames})
}

main().catch((e) => console.error(e,e.message));
```

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20231115/cfb0c917/attachment.htm>