[webkit-dev] Possible device scale factor emulation approaches?

[Adam Barth]

On Wed, Dec 5, 2012 at 10:44 PM, Alexander Pavlov <apavlov at chromium.org> wrote:
> On Thu, Dec 6, 2012 at 5:36 AM, Dana Jansens <danakj at chromium.org> wrote:
>> On Wed, Dec 5, 2012 at 8:28 PM, Adam Barth <abarth at webkit.org> wrote:
>> > On Wed, Dec 5, 2012 at 7:16 AM, Alexander Pavlov <apavlov at chromium.org>
>> > wrote:
>> >> I'm working on emulating the device scale factor with Web Inspector. My
>> >> goal
>> >> is to let web developers see their pages on "normal" (device pixel
>> >> ratio ==
>> >> 1) monitors the way they would look on e.g. retina screens.
>> >> Page::setDeviceScaleFactor() is not something we can use, since it will
>> >> have
>> >> the reverse effect when used by platform-specific compositor code
>> >> (which I
>> >> saw on Chromium, and its compositor is far from being ready for such
>> >> abnormal treatment.)
>> >
>> > I'm not sure I fully understand what you're trying to accomplish.  Is
>> > the developer using a high density screen or a low density screen?
>
> A low density screen is used (that's what I meant by "normal" (device pixel
> ratio == 1) monitors"). Imagine two 24" monitors, one high-density (H) and
> one low-density (L), standing next to each other. I want any page to be
> rendered exactly the same both on H and L monitor. So, @media and image set
> elements should be chosen as if L monitor were H, but then the respective
> images/glyphs/whatever downscaled to match the visible size of the same
> items in the H monitor.

I'm not sure I understand.  L has fewer pixels than H.  It's not clear
how two things can render exactly the same given that they'll render
using a different number of pixels.

>> Chromium has a command line option for setting a fake device pixel
>> ratio.  For example, if you're using a low density screen, you can
>> pass something like --device-pixel-ratio 2 to ask Chromium to render
>> as if your device has a high density display.  The net effect is that
>> the glyphs get really big (since they use 2x the pixels in each
>> dimension).
>>
>> > Alternatively, I can imagine that the developer is using a high
>> > density screen but wants to develop a web site that looks good on a
>> > low density screen.  In that case, they want to set a
>> > --device-pixel-ratio 1 but then render the web page pixel doubled
>> > (i.e., rasterizing each pixel as a 2x2 quad).  I don't think there's a
>> > command line option for that in Chromium, but it's something you might
>> > reasonably do in the compositor.
>
> Naturally, but I would prefer not to touch the compositor code, as it is
> currently converging to the hardware-accelerated Ash implementation, and
> it's going to take a while until the dust settles down.

That's not really a good reason to pick an inferior design.

>> > In general, though, trying to shim this feature into WebCore isn't
>> > likely the right approach.  For WebKit2, for example, both the UI
>> > process and the web process need to agree about the device pixel ratio
>> > or else the system gets confused about how big various regions of
>> > memory ought to be.  If you look at how we test high density rendering
>
> That's one reason why I do not want to fiddle with ACTUAL device pixel
> ratios (the same holds for the WebCore and compositor code in Chromium; the
> compositor code in desktop Chromium (non-Ash) is not ready to see anything
> other than 1 as device pixel ratio (WebViewImpl::setPageScaleFactor() has
> the following comment, "Don't allow page scaling when compositor scaling is
> being used, as they are currently incompatible.")).

What value do you want to return from window.devicePixelRatio?  If
you're going to return 1 on L but 2 on H, then the two are unlikely to
render similarly as JavaScript on the page is likely to branch (or
select subresources) based on that value.

>> > on low density machines, you'll see that we override the device's
>> > actual pixel density via the WebKit/WebKit2 API.  That's to avoid this
>> > confusion.
>
> Yes, this is exactly my scenario - render pages on low-density (DPR==1)
> monitors as if they were high-density (DPR==2 or something else.) So, if the
> device's actual pixel density is overridden in WebKit/WebKit2, will glyphs
> and images get 2x bigger than those without the override?
>
> My goal is to have them visually remain the same size (because 1 CSS pixel
> should still correspond to 1 device pixel, as DPR==1 for low-density
> screens) along with the @media rules and -webkit-image-set elements (and
> perhaps some other things) appropriate for DPR==2 taking effect (downscaling
> larger 2x DPR images where applicable, so that they will look as if they
> were rendered on a high-density screen.) Do you think this can be emulated
> through the page settings, same was as the touch emulation
> (WebCore::Settings::setTouchEventEmulationEnabled()) instead?

That's a super strange rendering mode.

>> >> I took the approach of instrumenting the WebCore/css and WebCore/page
>> >> code
>> >> relying on the page->deviceScaleFactor() value. This worked pretty
>> >> well, and
>> >> you can see the respective patch at
>> >> https://bugs.webkit.org/attachment.cgi?id=172046&action=prettypatch
>> >> (https://bugs.webkit.org/show_bug.cgi?id=100762), but now I'm wondering
>> >> if
>> >> there are better ways to implement this without instrumenting lots of
>> >> page->deviceScaleFactor() call sites.
>> >>
>> >> Comments, ideas, suggestions, please?
>> >
>> > If you want to see how this is done in Chromium, you might want to
>> > study the --device-pixel-ratio command line flag.  In general, I don't
>> > think you want to try to implementing this feature by hacking WebCore
>> > as it requires coordination beyond WebCore to do correctly.
>
> What if I try implementing this in a way that does not require any
> coordination beyond WebCore? I mean, overriding everything Web-related only
> in WebCore, but without requiring anything on the platform-specific part.

I don't think a WebCore-only implementation will be very high quality.
 It might be a convenient hack, but it's not the right way to
implement this feature.

Based on what you've described, it sounds like you should use
--force-device-scale-factor=2 and actually render to a 2x backing
store.  Then, once you've got the 2x backing store, you should
downsample it to match the actual resolution of the monitor.  I
suspect we do something like that in the multimonitor case when a
window is overlapping both a high density and a low density display.

>> I think Adam is refering to the --force-device-scale-factor=2 flag for
>> chromium. To try it out, you may need to build with the use_ash=1
>> flag.
>
> I've been in touch with aelias for over half a year, and his latest advice
> was to abstain from using this flag (or modifying
> page->setDeviceScaleFactor(), or touching the compositor :-) ) on something
> other than Ash for at least a few quarters more, until the Chromium
> compositor code converges to the implementation used by Ash (that's what I
> implied by saying that "its compositor is far from being ready for such
> abnormal treatment"). Until then, event-handling code will still handle
> physical rather than emulated mouse pointer coordinates, so the page
> interaction will be completely broken, and you will see lots of
> noise/garbage rendered where inapplicable.)

That might mean you should hold off on this feature until the
compositor is able to support it.  Hacking this into WebCore isn't
going to produce a high quality implementation.

Adam