[Webkit-unassigned] [Bug 260460] New: Slow performance when combining pushing ImageBitmaps to VRAM from a video source

bugzilla-daemon at webkit.org bugzilla-daemon at webkit.org
Sun Aug 20 22:26:28 PDT 2023


            Bug ID: 260460
           Summary: Slow performance when combining pushing ImageBitmaps
                    to VRAM from a video source
           Product: WebKit
           Version: Safari 16
          Hardware: iPhone / iPad
                OS: iOS 16
            Status: NEW
          Severity: Major
          Priority: P2
         Component: WebGL
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: webkit at frost.kiwi
                CC: dino at apple.com, kbr at google.com, kkinnunen at apple.com

iOS struggles when pushing ImageBitmaps created from video sources to VRAM via WebGL. I have a live demo with debug timers, where you can see this: https://mirrorball.frost.kiwi/ with the source code being https://github.com/FrostKiwi/Mirrorball .
In the demo, images and videos are converted via https://developer.mozilla.org/en-US/docs/Web/API/createImageBitmap to ImageBitmaps, the modern object that is supposed to allow performant interaction with WebGL.

For static images, image Bitmaps are created here: https://github.com/FrostKiwi/Mirrorball/blob/3b3b9426f8935f1e1a7980a770f5fb97c723fe86/src/js/media.js#L261 and pushed with a brand new created texture to VRAM here: https://github.com/FrostKiwi/Mirrorball/blob/3b3b9426f8935f1e1a7980a770f5fb97c723fe86/src/js/media.js#L276

For videos, a different path is chosen to only update the texture, not recreate it. The bitmap creation happens here: https://github.com/FrostKiwi/Mirrorball/blob/3b3b9426f8935f1e1a7980a770f5fb97c723fe86/src/js/main.js#L88 and the update to the texture via texSubImage2D() happens here: https://github.com/FrostKiwi/Mirrorball/blob/3b3b9426f8935f1e1a7980a770f5fb97c723fe86/src/js/media.js#L303C9-L303C22

To see the resulting performance, you can go in the controls side menu -> Settings -> Debug -> Eruda Debug console. This enables measurement and printing of time taken to create the ImageBitmaps and the upload to VRAM. It also allows one to see the timers on iOS devices without debug consoles via https://github.com/liriliri/eruda , the small square icon appearing on the bottom of the page.

Choosing any video from the media gallery results in the following performance:
On an iPad 9th Gen, iOS Version 16.6: ~5ms to create the ImageBitmap (excellent), ~36ms to update an already existing 2048²px texture via texSubImage2D(). This unfortunately tanks all performance below 15fps. Not only that, it also makes input delay skyrocket. Even though the Video's framerate is supposed to be decoupled from the WebApps WebGL renderloop, the upload takes so long as to make the whole interface a laggy experience.

This is strange, since weak and outdated non-iOS devices do not struggle in the same manner.

This issue potentially corroborates these issues:

You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20230821/c47f4538/attachment.htm>

More information about the webkit-unassigned mailing list