[Webkit-unassigned] [Bug 90375] Parallel image decoders

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=90375





--- Comment #78 from Huang Dongsung <luxtella at company100.net>  2012-08-10 05:22:35 PST ---
(In reply to comment #77)
> I'm a little bit worried about the rendering behavior of the parallel image decoding as proposed here. I haven't interacted with a compiled binary containing these changes, but here is what I expect it to behave like. luxtella or skyul, can you respond to the following analysis?

Very good questions! I answered each question in the below. If you think my answer is not detailed enough, please let me know.

>  - The first attempt to draw a BitmapImage essentially sets up a race between the parallel decode, and the draw operation's attempt to read the frame.

There is no race at all. The parallel image decoder does not access the frame being read by the draw operation. It sends segments of decoded data to the main thread, then the main thread creates a frame by appending these segments.

Please refer to 3.6 Data transfers in the design document.
https://docs.google.com/document/pub?id=12gf7MhNHfupeR3GdRF-h2Vzg86viCbwXq8_JByHKlg0

>  - Given the timings involved, the draw operation will likely proceed before the decode operation really starts in earnest.

If the parallel image decoder cannot create a frame, ImageSource returns 0 to BitmapImage.  BitmapImage::draw() skips drawing in this case.

void BitmapImage::draw(GraphicsContext* ctxt, const FloatRect& dstRect,
                       const FloatRect& srcRect, ColorSpace colorSpace, CompositeOperator compositeOp)
{
    ...
    NativeImageSkia* bm = nativeImageForCurrentFrame();
    if (!bm)
        return; // It's too early and we don't have an image yet.
    ...
}

>  - Usually, we'll expect the draw operations to complete next frame.

The parallel image decoder does not handle multi-frame images because the gain is small while the complexity is so big. There is no behavior change in this case. 

>  - As a result, we'll visually see a flash of a transparent region while waiting for the image data to become available. That flash is arguably justifiable (?) since images are expected to load progressively anyway.

My feeling is that the parallel image decoder does not seem to worsen a flash of a transparent region compared to the current implementation. However, I found one particular case which indeed causes flashing. I'l will explain more on this at the end of this comment.

>  - If an <img>'s data is available immediately (e.g. the compressed image data is in cache because it was prefetched via XHR), the race parallel decode will typically add a frame of latency to the presentation of the image.

Even if an <img>'s data is available, the parallel image decoder can give us benefits. For example, when encoded data of all images are cached, loading the following site still causes about 1 sec UI hang-up. (http://www.dorothybrowser.com/test/webkitTest/imgdecode/bgimage-png/test.html) But I couldn't feel any hang-up after applying the parallel image decoder.

If ImageSource has the decoded frame, it simply returns the decoded frame without triggering parallel image decoding.


>  - If the MemoryCache is under extreme pressure and thrashing, what used to be a performance problem (slower paint operations because allocation/decoding happens each draw operation) will devolve to a kind of nondeterministic behavior (some timing-dependent subset of decoded images -- possibly all images -- are never available at draw time, because the parallel decode work is evicted before it's consumed in the next frame).

Good question! We fixed Bug 90721 to avoid this problem. After Bug 90721, MemoryCache no longer removes the decoded data of CahcedImage if image will be rendered soon. So the parallel image decoding work is never evicted if clients need to render the image.

> 
> Incurring flashing or an extra frame of latency to the visibility of every new <img> element, this does not seem desirable if serial decoding would only take a small fraction of the frame time.

As I mentioned above, I don't think the parallel image decoder worsens the situation here. However, I found your concern makes sense in the following site: http://www.dorothybrowser.com/test/webkitTest/imgdecode/gmarket/kiru.kr/kiru/html/singlelongopst.htm

After applying parallel image decoding, we could notice flashing when we put the mouse cursor over images. When I saw this at the first time, I was very curious why the image in the back was flashing.  After I inspected the site, I realized that the images are not just two images: one transparent image in the front and the other in the back. They are one normal image and the other hover image. 

When we put the mouse over these images, the normal image is set to 'display:hidden' and the hover image is set to 'display:block'. So while the parallel image decoder is decoding the hover image, WebKit renders a white background instead. This causes flashing. If the site developer showed the transparent image in the front on top of the image in the back, flashing would've disappeared. This is one rare case where parallel image decoder actually causes flashing. I expect we won't encounter this kind of flashing in most sites.

I want to listen opinions from all folks whether this behavior is acceptable or not.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.