[webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Noam Rosenthal]

On Mon, Nov 11, 2019 at 10:13 PM Maciej Stachowiak <mjs at apple.com> wrote:

>
>
> On Nov 11, 2019, at 4:37 AM, Noam Rosenthal <noam at webkit.org> wrote:
>
>
>
> On Mon, Nov 11, 2019 at 12:13 AM Maciej Stachowiak <mjs at apple.com> wrote:
>
>> - Is there a specification for Content-DPR? All I could find in bugzila
>> and elsewhere is links to <
>> https://httpwg.org/http-extensions/client-hints.html>, which does not
>> mention Content-DPR at all).
>>
> In a nutshell, the spec is in transition from IETF to HTML/fetch, and the
> client-hints aspects of it are still in progress (unlike conent-dpr, which
> is much simpler hasn't changed since introduced). It's documented here:
> https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md
> .
> I will answer more about this separately if required, waiting for some
> info from the people working on the spec originally.
>
>
> It looks like these are the relevant Pull Requests:
>
>
> HTML: https://github.com/whatwg/html/pull/3774 |
> https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density
> Fetch: https://github.com/whatwg/fetch/pull/773 |
> https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density
>
> It looks like these are in a somewhat messy state. For example, Fetch
> places the Content-DPR value in an “image density” variable, but it doesn’t
> look like the Pull Request to HTML doesn’t use it anywhere. As another
> example, HTML directly references Content-DPR as setting the “current pixel
> density”, but does not specify that it would take priority over a density
> derived from srcset. There are also no diffs to CSS Images or to SVG, which
> provide distinct ways to reference images and which presumably should
> respect Content-DPR.
>



>
>
>
>> - Can you give us some examples of how the CDN would make the decision of
>> what density of image to serve without knowing the client’s DPR via
>>  client-hints?
>>
>  Some examples from  https://github.com/eeeps/content-dpr-explainer:
> - making decisions by user agent, like choosing to cap certain images for
> user-agents known to have smaller displays
> - making decisions by traffic/geo, like serving lower-resolution images
> when traffic is bogged down
> - A client may ask for "placeholder image" (e.g. ?placeholder=true in the
> URL), and the CDN would decide whether to serve a lower-quality JPEG or to
> scale it down, or do both.
>
>
> These seem like reasonable use cases.
>
>
> - Is this presuming situations where the CDN serves the images but not the
>> markup or CSS (which presumably could be rewritten to account for DPR)?
>>
> Correct.
>
>
>> - What happens if Content-DPR is provided, but the markup or CSS has
>> conflicting explicit info? For example, <img srcset=“image-2x.jpg 2x,
>> image.jpg 1x”>, and image-2x.jpg is served with a Content-DPR header of 3x.
>> Or the analogous thing with CSS.
>>
> When image size is decided, css/markup takes precedence, then content DPR,
> and finally srcset-selection DPR. This is actually a bonus value of this
> feature - if the server happens to serve an image that differs from the
> ratio requested in the srcset tag, the intrinsic size match the served
> content and not the request (which is more of a recommendation).
>
> - Does Content-DPR have any effect on images where an explicit size is
>> provided, either in markup or in CSS? (I’m assuming no.)
>>
> No, it only has effect on intrinsic size.
>
>
> Overall, this seems like a feature with valid use cases. But
> unfortunately, it’s a (currently) Blink-only feature with no clear
> specification. To the extent there is a spec, it’s mixed in with multiple
> Pull Requests. These pull requests mix in Client Hints, a feature unlikely
> to gain multi implementor support, so they probably won’t land any time
> soon. And the language about Content-DPR is broken and does not not seem to
> match Chrome’s actual implementation.
>
> So in summary, there is no proper spec, and Chrome has shown willingness
> to have their implementation change faster than even draft spec language in
> PRs can keep up with.
>

> Although the use case seems valid, I don’t think it’s a good idea for
> WebKit to implement the feature in this state. It seems likely to cause
> interop headaches and a need to reverse engineer Blink rather than
> developing against specs and test cases.
>
> There has been a lot of work done on the spec front since the comments
above, and I think we're ready for a re-review the state.
* The next pull request: https://github.com/whatwg/html/pull/5112 has been
thumbed up to make it into the HTML spec, and is separate from
client-hints, but requires second implementor interest, which is what this
email thread is for :)
* Web platform tests are comprehensive, separated from client-hints, and
match both the spec and the Chrome implementation:
https://github.com/web-platform-tests/wpt/tree/master/content-dpr. We can
always add more if we find more use-cases.

Would love to hear more thoughts!

Thanks
Noam
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20200303/7e371f34/attachment.htm>