[webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer

[Maciej Stachowiak]

> On Mar 2, 2020, at 11:22 PM, Noam Rosenthal <noam at webkit.org> wrote:
> 
> 
> 
> On Mon, Nov 11, 2019 at 10:13 PM Maciej Stachowiak <mjs at apple.com <mailto:mjs at apple.com>> wrote:
> 
> 
>> On Nov 11, 2019, at 4:37 AM, Noam Rosenthal <noam at webkit.org <mailto:noam at webkit.org>> wrote:
>> 
>> 
>> 
>> On Mon, Nov 11, 2019 at 12:13 AM Maciej Stachowiak <mjs at apple.com <mailto:mjs at apple.com>> wrote:
>> - Is there a specification for Content-DPR? All I could find in bugzila and elsewhere is links to <https://httpwg.org/http-extensions/client-hints.html <https://httpwg.org/http-extensions/client-hints.html>>, which does not mention Content-DPR at all).
>> In a nutshell, the spec is in transition from IETF to HTML/fetch, and the client-hints aspects of it are still in progress (unlike conent-dpr, which is much simpler hasn't changed since introduced). It's documented here: https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md <https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md>.
>> I will answer more about this separately if required, waiting for some info from the people working on the spec originally.
> 
> It looks like these are the relevant Pull Requests:
> 
> 
> HTML: https://github.com/whatwg/html/pull/3774 <https://github.com/whatwg/html/pull/3774> | https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density <https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density>
> Fetch: https://github.com/whatwg/fetch/pull/773 <https://github.com/whatwg/fetch/pull/773> | https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density <https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density>
> 
> It looks like these are in a somewhat messy state. For example, Fetch places the Content-DPR value in an “image density” variable, but it doesn’t look like the Pull Request to HTML doesn’t use it anywhere. As another example, HTML directly references Content-DPR as setting the “current pixel density”, but does not specify that it would take priority over a density derived from srcset. There are also no diffs to CSS Images or to SVG, which provide distinct ways to reference images and which presumably should respect Content-DPR.
> 
>  
> 
>>  
>> - Can you give us some examples of how the CDN would make the decision of what density of image to serve without knowing the client’s DPR via  client-hints? 
>>  Some examples from  https://github.com/eeeps/content-dpr-explainer <https://github.com/eeeps/content-dpr-explainer>:
>> - making decisions by user agent, like choosing to cap certain images for user-agents known to have smaller displays
>> - making decisions by traffic/geo, like serving lower-resolution images when traffic is bogged down
>> - A client may ask for "placeholder image" (e.g. ?placeholder=true in the URL), and the CDN would decide whether to serve a lower-quality JPEG or to scale it down, or do both.
> 
> These seem like reasonable use cases.
> 
>> 
>> - Is this presuming situations where the CDN serves the images but not the markup or CSS (which presumably could be rewritten to account for DPR)?
>> Correct.
>>  
>> - What happens if Content-DPR is provided, but the markup or CSS has conflicting explicit info? For example, <img srcset=“image-2x.jpg 2x, image.jpg 1x”>, and image-2x.jpg is served with a Content-DPR header of 3x. Or the analogous thing with CSS.
>> When image size is decided, css/markup takes precedence, then content DPR, and finally srcset-selection DPR. This is actually a bonus value of this feature - if the server happens to serve an image that differs from the ratio requested in the srcset tag, the intrinsic size match the served content and not the request (which is more of a recommendation).
>> 
>> - Does Content-DPR have any effect on images where an explicit size is provided, either in markup or in CSS? (I’m assuming no.)
>> No, it only has effect on intrinsic size. 
> 
> Overall, this seems like a feature with valid use cases. But unfortunately, it’s a (currently) Blink-only feature with no clear specification. To the extent there is a spec, it’s mixed in with multiple Pull Requests. These pull requests mix in Client Hints, a feature unlikely to gain multi implementor support, so they probably won’t land any time soon. And the language about Content-DPR is broken and does not not seem to match Chrome’s actual implementation.
> 
> So in summary, there is no proper spec, and Chrome has shown willingness to have their implementation change faster than even draft spec language in PRs can keep up with. 
> 
> Although the use case seems valid, I don’t think it’s a good idea for WebKit to implement the feature in this state. It seems likely to cause interop headaches and a need to reverse engineer Blink rather than developing against specs and test cases.
> 
> There has been a lot of work done on the spec front since the comments above, and I think we're ready for a re-review the state.
> * The next pull request: https://github.com/whatwg/html/pull/5112 <https://github.com/whatwg/html/pull/5112> has been thumbed up to make it into the HTML spec, and is separate from client-hints, but requires second implementor interest, which is what this email thread is for :)

I don’t think it has? I’m seeing:
"[ ]  At least two implementers are interested (and none opposed):”

And satisfying this is a requirement for adding features to WHATWG Living Standards.

And the review block at the bottom says “Review required” and “Merging is Blocked with big red X’s.

> * Web platform tests are comprehensive, separated from client-hints, and match both the spec and the Chrome implementation: https://github.com/web-platform-tests/wpt/tree/master/content-dpr <https://github.com/web-platform-tests/wpt/tree/master/content-dpr>. We can always add more if we find more use-cases.
> 
> Would love to hear more thoughts!

Sounds like a good time to re-review.

Regards,
Maciej

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20200304/36887e6e/attachment.htm>