[Webkit-unassigned] [Bug 206636] New: getUserMedia video stream to texture sometimes gives old frames.

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=206636

            Bug ID: 206636
           Summary: getUserMedia video stream to texture sometimes gives
                    old frames.
           Product: WebKit
           Version: Safari 13
          Hardware: Unspecified
                OS: iOS 13
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Media
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: nb at 8thwall.com

Created attachment 388497

  --> https://bugs.webkit.org/attachment.cgi?id=388497&action=review

source code for repro case

Sometimes calling 

gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, video)

populates the texture with frame data that is old, usually 2-3 frames old but sometimes 15 or more.  When drawing these textures to a canvas, this gives an impression that the video feed is jumping, because a scene from a while ago will flash momentarily.

The effect is rare in normal conditions but can be reliably triggered by extending the length of an animation frame by a long random interval, e.g. 20ms-200ms.

The effect can be mitigated by skipping every other animation frame, i.e. 

const nativeRaf = window.requestAnimationFrame
window.requestAnimationFrame = (fn) => nativeRaf(() => nativeRaf((timestamp) => fn(timestamp)))

A repro case can be viewed on an iphone here: 8th.io/ft4h9

The repro app does the following:
* getUserMedia -> stream -> video
* video -> texture (there is a rotating pool of 10 textures)
* texture -> luminance shader -> framebuffer -> readpixels -> uint8array
* texture -> canvas -> window
* uint8array -> frame hash code based on pixel content -> frame hash history
* search frame hash history to find duplicate hashes
* if there is a duplicate hash, print the number of frames ago for the duplicate frame pixels (do this for the 5 most recent duplicates), and increment a counter for the number of duplicate frames.

We tested this with an iPhone11 Pro with iOS 13.3.

Source code for the repro is attached.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20200123/6c81d6d7/attachment-0001.htm>