[webkit-dev] Status of multithreaded image decoding

[Dong Seong Hwang]

2012/8/14 KwangYul Seo <skyul at company100.net>:
>
>
> On Tue, Aug 14, 2012 at 4:03 AM, Maciej Stachowiak <mjs at apple.com> wrote:
>>
>>
>> On Aug 13, 2012, at 11:08 AM, Alpha Lam <hclam at chromium.org> wrote:
>>
>> That's a good point. I'm not sure but a safe bet would be after RenderView
>> is layout'ed then iterate through images to start decoding.
>>
>> The same thing would be needed for scrolling too, as page scrolls need to
>> iterate images in the viewport.
>>
>>
>> This approach is probably safe (as far as I know) but would have the
>> downside of an extra pass over the whole render tree, or else overhead of
>> maintaining an up-to-date list of rendered images; and it would happen very
>> close to painting.
>>
>
> Yes, the problem is that we don't have much time between layout and painting
> because painting is usually  immediately followed by layout. I'm skeptical
> about the gain we could get by triggering image decoding in the layout
> phase, but I'd like to defer the conclusion until we have real numbers.

Trigger image decoding early in layout and scroll will certainly
relieve flashing though it can’t completely remove the problem. We
will apply this optimization to parallel image decoder.

Please note that this optimization is applied only to parallel image
decoder, not Google’s deferred image decoder.

Thank all for good information.

Regards,
DS Huang

>
>> I think to some extent, benefit from parallelism is in direct tension with
>> benefit from lazy on-demand decoding.
>>
>
> Unfortunately, yes. Because of the lazy nature of image decoding, we don't
> have much time for decoding before images need to be painted. So we skip
> image decoding (and just trigger decoding in the background) in the first
> paint pass and update the decoded images later.
>
>>
>> Note by the way that in general many kinds of render objects can result in
>> rendering images, not just RenderImage. Consider the various forms of CSS
>> images (CSS backgrounds, CSS content property pointing to an image, etc).
>>
>
> We made CSS background, border and mask images be decoded by parallel image
> decoders.
>
> https://bugs.webkit.org/show_bug.cgi?id=91203
>
>>
>> Regards,
>> Maciej
>>
>>
>> 2012/8/13 Alpha (Hin-Chung) Lam <hclam at google.com>
>>>
>>> That's a good point. I'm not sure but a safe bet would be after
>>> RenderView is layout'ed then iterate through images to start decoding.
>>>
>>> The same thing would be needed for scrolling too, as page scrolls need to
>>> iterate images in the viewport.
>>>
>>> Alpha
>>>
>>> 2012/8/13 Maciej Stachowiak <mjs at apple.com>
>>>>
>>>>
>>>> The thing I'm not confident of is whether an image's position in
>>>> absolute coordinates can be changed by an ancestor after RenderImage::layout
>>>> completes. It would be helpful if a layout expert would weigh in.
>>>>
>>>>  - Maciej
>>>>
>>>> On Aug 13, 2012, at 3:20 AM, Dong Seong Hwang <luxtella at company100.net>
>>>> wrote:
>>>>
>>>> > https://bugs.webkit.org/show_bug.cgi?id=90375#c80
>>>> >
>>>> > In the above link, Hin-Chung shows how to determine whether an image
>>>> > is actually painted.
>>>> >
>>>> > 2012/8/13 Maciej Stachowiak <mjs at apple.com>:
>>>> >> I that case, starting async decoding at layout time makes sense if
>>>> >> and only
>>>> >> if at layout to e you can predict what you will paint. I don't know
>>>> >> enough
>>>> >> about our rendering to know of that is the case.
>>>> >>
>>>> >> - Maciej
>>>> >>
>>>> >>
>>>> >>
>>>> >> On Aug 12, 2012, at 5:34 PM, Peter Kasting <pkasting at chromium.org>
>>>> >> wrote:
>>>> >>
>>>> >> On Sun, Aug 12, 2012 at 1:24 PM, Maciej Stachowiak <mjs at apple.com>
>>>> >> wrote:
>>>> >>>
>>>> >>> Why not start asynchronous decoding immediately as the image is
>>>> >>> loading,
>>>> >>> and synchronize at paint time? What is the benefit of waiting until
>>>> >>> layout
>>>> >>> time to start decoding the image data?
>>>> >>
>>>> >>
>>>> >> Uninformed guess (since I haven't touched the decoders in a while),
>>>> >> but
>>>> >> currently we don't decode unless the image is actually painted, which
>>>> >> helps
>>>> >> a ton on pages that are an enormous long string of images (common
>>>> >> cases:
>>>> >> Boston Big Picture blog, various porn sites), since most of the
>>>> >> images can
>>>> >> be decoded after initial layout, or not at all (if the user never
>>>> >> scrolls
>>>> >> down enough).  If we started decoding as images loaded I'd imagine
>>>> >> we'd do
>>>> >> (possibly a lot of) extra work compared to today.
>>>> >>
>>>> >> PK
>>>> >>
>>>> >>
>>>> >> _______________________________________________
>>>> >> webkit-dev mailing list
>>>> >> webkit-dev at lists.webkit.org
>>>> >> http://lists.webkit.org/mailman/listinfo/webkit-dev
>>>> >>
>>>>
>>>> _______________________________________________
>>>> webkit-dev mailing list
>>>> webkit-dev at lists.webkit.org
>>>> http://lists.webkit.org/mailman/listinfo/webkit-dev
>>>
>>>
>>
>>
>>
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo/webkit-dev
>>
>
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo/webkit-dev
>