[Webkit-unassigned] [Bug 272521] New: [MSE] Decoding errors when appending segment with B-frames and previous segment ends with an I-frame

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=272521

            Bug ID: 272521
           Summary: [MSE] Decoding errors when appending segment with
                    B-frames and previous segment ends with an I-frame
           Product: WebKit
           Version: WebKit Nightly Build
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Media
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: cadubentzen at igalia.com

When a segment containing B-frames is appended (i.e. with out-of-display-order frames, and samples PTS and DTS values don't always match), WebKit has logic to erase frames from the previous segment and avoid decoding glitches (see bug 206412). However, the current logic falls short in one edge case: if the previous segment ends with an I-frame in the overlap region to be erased.

If the demuxer honors ISOBMFF edit lists, then the first I-frame from an incoming segment with B-frames can be placed earlier in decoding order than the last (I-)frame from the previous segment (potentially with different resolution), and that last I-frame doesn't get erased, and will be pushed for decoding. This confuses the decoder and the following P/B frames will fail to decode or decode incorrectly with artifacts due to missing/incorrect reference frame.

When this edge case is triggered:
- In Apple WebKit's, playback directly stops and we get an error event.
- In upstream GStreamer-based ports we don't have this problem due to ignoring edit lists (see bug 231019). But in downstream WPE ports where it's enabled, we get macroblock artifacts, but could potentially get fatal errors as well depending on the decoder plugin that's used.

Hosted test page to reproduce it manually: https://people.igalia.com/cadubentzen/webkit/out-of-order (EXPECTED: playback failure on Safari at 4s. For GStreamer-based ports, not reproducible upstream due to ignoring edit lists). It works with Firefox and Chromium.

A detailed analysis was done from a case that triggered this in https://github.com/WebPlatformForEmbedded/WPEWebKit/issues/1301 and https://github.com/WebPlatformForEmbedded/WPEWebKit/pull/1184. Credits to Vivek Arumugam.

I have patch with a possible fix and a layout test that reproduces this with `video/mock`, just need a bit more checking before opening for review. The patch is still only a mitigation as this scenario isn't well defined in the spec (https://github.com/w3c/media-source/issues/187).

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20240411/59d1ab02/attachment.htm>