On Wed, Sep 21, 2011 at 10:44 AM, Alexey Proskuryakov <ap@webkit.org> wrote:

This API seems clearly necessary to implement traditional FPS game controls. On the other hand, I would not want my browser to have this kind of functionality. Arguably the main benefit of browsers is that they cannot do many things, and are thus relatively safe to use, not yet triggering the sort of fear normally associated with installing software on desktop platforms. Interfering with mouse movement would be one of the most dangerous and frustrating things Web pages could possibly do. The specification draft acknowledges the problem of inability to interact with user agent and the rest of the OS in mouse lock mode. However, the suggested mitigations seem fairly weak. Users are not trained to hit Esc anytime their computer stops properly reacting to their actions - I certainly wouldn't think of doing that. A permanent reminder would be extremely annoying, and we cannot expect that it would be read anyway. Fullscreen is one mode where browsers are expected to take more control over user interaction. Perhaps mouse lock could be appropriate in fullscreen. However, regular fullscreen applications do not use a mouse lock, so users are not trained to escape it - one can always move the mouse pointer to top of screen to get back their menu bar.

Is that a Mac thing? Mousing around in a fullscreen flash app on Linux or Windows 7 certainly doesn't pop up a menu bar when I hit the top. And the way out is always to hit ESC [although there's often a button as well, depending on the application], so I'm not sure what the problem with fullscreen mouse lock would be.

The only related case I can think of is emulators, which release the mouse on certain modifier keys, such as Cmd or Ctrl+Option. So, I'm not sure how a safe UI would work even in this case. I'm curious about the following provision in the spec:

Mouse events normally considered user gestures (e.g. mousedown, click) for security purposes (such as when allowing pop-up windows) should not be treated as user gestures when under mouse lock to prevent malicious cross origin click redirecting.

My understanding is that when browser is in control, the primary security concern is with making the user believe that they are interacting with a different site, and thus stealing confidential data the user types (especially passwords). Displaying a pop-up window sounds like a very minor issue when malicious code is already in main frame, can draw anything, and can control mouse movement. Is there a specific attack scenario where this limitation helps? - WBR, Alexey Proskuryakov 20.09.2011, в 20:53, Vincent Scheib написал(а):

I have proposed Mouse Lock to be adopted by the W3 WebEvents Working Group: http://lists.w3.org/Archives/Public/public-webevents/2011JulSep/0064.html http://lists.w3.org/Archives/Public/public-webapps/2011JulSep/1424.html

On Mon, Sep 12, 2011 at 4:50 PM, Vincent Scheib <scheib@chromium.org> wrote:

...

A Mouse Lock API has been under discussion on the W3 public-webapps list "Mouse Lock"(1) and earlier "Mouse Capture for Canvas"(2). The primary goal is to enable use cases such as first person perspective 3D applications. This is done by providing scripted access to mouse movement data while locking the target of mouse events to a single element and removing the cursor from view. I have been working to formalize the discussions into a draft specification: http://goo.gl/9G8pd, and have a work in progress prototype for WebKit | Chrome. I will publish a WIP patch when it is ready for more eyes. It will be developed behind a compile (and runtime) time flag. I'd like to invite any specification discussion to the Mouse Lock thread (1), and WebKit implementation discussion here. Cheers, -Vince --

1. http://lists.w3.org/Archives/Public/public-webapps/2011JulSep/thread.html#ms...

2. http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/thread.html#ms... W3 issue: http://www.w3.org/Bugs/Public/show_bug.cgi?id=9557 Firefox issue: https://bugzilla.mozilla.org/show_bug.cgi?id=633602 Chrome issue: http://code.google.com/p/chromium/issues/detail?id=72754

_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

On Wed, Sep 21, 2011 at 11:11 AM, Alexey Proskuryakov <ap@webkit.org> wrote:

21.09.2011, в 10:56, Eric Uhrhane написал(а):

...

...

one can always move the mouse pointer to top of screen to get back their menu bar. Is that a Mac thing?

Yes, this is how fullscreen applications regularly work on OS X.

This is not true for several Flash-based fullscreen video players that I've used on my mac. The interaction there is that upon entering fullscreen Flash displays an overlay indicating that hitting ESC will exit fullscreen mode and the video player displays its controls near the bottom of the screen. After these fade away, the video controls appear only when moving the mouse pointer to the bottom ~1/3rd of the screen. Moving the mouse pointer to the top of the screen does nothing. Hitting ESC or clicking on the appropriate button in the video player's controls exits fullscreen mode. This seems entirely reasonable to me, the keyboard control is provided by Flash itself to prevent bad SWFs from taking control of my computer and the SWF itself provides the additional controls that make sense for its domain. - James

...

Mousing around in a fullscreen flash app on Linux or Windows 7 certainly doesn't pop up a menu bar when I hit the top. And the way out is always to hit ESC [although there's often a button as well, depending on the application], so I'm not sure what the problem with fullscreen mouse lock would be.

I do not have recent knowledge of Linux, but my understanding is that Windows UI is undergoing a major redesign with Metro, so basing APIs largely on how Windows used to work up to version 7 may not be future proof.

- WBR, Alexey Proskuryakov

_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

On Wed, Sep 21, 2011 at 12:05 PM, Alexey Proskuryakov <ap@webkit.org> wrote:

21.09.2011, в 11:21, James Robinson написал(а):

...

...

one can always move the mouse pointer to top of screen to get back their menu bar. Is that a Mac thing?

Yes, this is how fullscreen applications regularly work on OS X.

This is not true for several Flash-based fullscreen video players that I've used on my mac. The interaction there is that upon entering fullscreen Flash displays an overlay indicating that hitting ESC will exit fullscreen mode and the video player displays its controls near the bottom of the screen. After these fade away, the video controls appear only when moving the mouse pointer to the bottom ~1/3rd of the screen. Moving the mouse pointer to the top of the screen does nothing. Hitting ESC or clicking on the appropriate button in the video player's controls exits fullscreen mode. This seems entirely reasonable to me, the keyboard control is provided by Flash itself to prevent bad SWFs from taking control of my computer and the SWF itself provides the additional controls that make sense for its domain.

You are describing how Flash exits full screen, and the fact that it apparently always has mouse lock in full screen (so you cannot get to browser menu bar by moving mouse pointer up). It's a different behavior from what this API provides.

Besides being a security measure for a subtly different feature, Flash behavior is quite annoying, and not very efficient - see e.g. < http://www.bunnyhero.org/2008/05/10/scaring-people-with-fullscreen/>.

To demonstrate the difference with Flash, there is no mouse lock in Safari fullscreen, so you can move the mouse pointer up to get the menu bar.

Anyway, I'm not sure if we already agreed that mouse lock is only desirable in full screen. I think that the spec has the goal of enabling it in browser windows.

21.09.2011, в 11:22, Darin Fisher написал(а):

basing APIs largely on how Windows used to work up to version 7 may not be

...

future proof.

Yes, but >90% of internet users are not familiar with Metro. They are familiar with Windows as it exists today (XP thru 7).

More than 90% is an understatement :)

Is your implication that Microsoft will be forced to make future Windows UI behave the same as Windows 7 does? Otherwise, it's not clear to me how a reference to old behavior is relevant to being future proof.

My point was that emulating the behavior that most users are familiar with will likely create a predictable user experience. Most users are familiar with hitting ESC to exit fullscreen, so it seems sensible to piggyback on that. By the way, wouldn't it be better to debate this API in the WebEvents WG? -Darin

On Wed, Sep 21, 2011 at 12:46 PM, Peter Kasting <pkasting@chromium.org>wrote:

On Wed, Sep 21, 2011 at 12:05 PM, Alexey Proskuryakov <ap@webkit.org>wrote:

...

Anyway, I'm not sure if we already agreed that mouse lock is only desirable in full screen. I think that the spec has the goal of enabling it in browser windows.

Indeed, this is explicitly a use case we want to allow. It's reasonable to want to play many mouse-lock-requiring games (Quake, Starcraft, etc.) in non-fullscreen mode, e.g. so one can still keep some other critical display open elsewhere on the screen.

PK

True, although there are still questions about how the user experience will work for non-fullscreen. I think we only feel confident that we have a decent fullscreen solution at this point, and the plan was to restrict the initial implementation to fullscreen mode for that reason. -Darin

On Wed, Sep 21, 2011 at 1:15 PM, Vincent Scheib <scheib@chromium.org> wrote:

Re: Alexey Proskuryakov's comment:

...

I'm curious about the following provision in the spec:

...

Mouse events normally considered user gestures (e.g. mousedown, click) for security purposes (such as when allowing pop-up windows) should not be treated as user gestures when under mouse lock to prevent malicious cross origin click redirecting. My understanding is that when browser is in control, the primary security concern is with making the user believe that they are interacting with a different site, and thus stealing confidential data the user types (especially passwords). Displaying a pop-up window sounds like a very minor issue when malicious code is already in main frame, can draw anything, and can control mouse movement. Is there a specific attack scenario where this limitation helps?

The concern I was attempting to address is primarily that of click jacking. E.g. Preventing a malicious top level page from directing unintended clicks to an iframe with a target site (e.g. a bank). Upon reflection, perhaps the correct way to limit this is to require the target element to match the top level origin as well. Then user gestures can only be delivered to the top level origin.

I wouldn't worry too much about that. A site can already mount a clickjacking attack without using mouselock. It's not clear to me that mouselock make the situation any worse than it is already. Adam

Re: Mouse lock can be a nuisance. Yes, it can, but it also is essential for many compelling applications and user interactions. We believe user-agents can find a balance of providing that ability and diminishing it's abuse. The example of Flash's abusable full screen mode is relevant in that I believe the amount of users enjoying the feature vastly outweigh the amount of nuisance caused. We'll seek to find the appropriate ease of use and reduction of nuisance, but in the extreme can fall back onto heavy weight measures such as installing a web-app or prompting for permission first. The API is designed to tolerate these methods (asynchronous success/failure). _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

On Thu, Sep 22, 2011 at 2:23 PM, Alexey Proskuryakov <ap@webkit.org> wrote:

21.09.2011, в 12:52, Darin Fisher написал(а):

True, although there are still questions about how the user experience will work for non-fullscreen. I think we only feel confident that we have a decent fullscreen solution at this point, and the plan was to restrict the initial implementation to fullscreen mode for that reason.

Would it make sense to simplify the API too? Parts of it seem unnecessary if mouse lock only works in full screen - all additions to MouseEvent in particular can likely be dropped. Security considerations are also substantially different.

It might be more productive to treat fullscreen mouse lock as a complete API, with only a potential for being some day extended to support windowed mode.

- WBR, Alexey Proskuryakov

Our next step is to extend this to non-fullscreen, so I think it would be counter-productive to preclude non-fullscreen at this time. -Darin

On Thu, Sep 22, 2011 at 2:38 PM, Alexey Proskuryakov <ap@webkit.org> wrote:

22.09.2011, в 14:25, Darin Fisher написал(а):

Our next step is to extend this to non-fullscreen, so I think it would be counter-productive to preclude non-fullscreen at this time.

I think that adding an API for non-fullscreen mouse lock would be a detriment to the Web platform. So gating fullscreen API on it seems pretty controversial.

- WBR, Alexey Proskuryakov

Would you mind raising your concerns with the working group? I believe there is a fair bit of interest in providing mouse lock in the non-fullscreen case too. I can imagine it may require some additional UI. It makes sense to me to design the mouse lock API with the idea that it could work in either fullscreen or non-fullscreen mode. Then it just becomes a question of policy and UI as to whether it runs in neither, one or both. -Darin

On Wed, Sep 21, 2011 at 11:11 AM, Alexey Proskuryakov <ap@webkit.org> wrote:

21.09.2011, в 10:56, Eric Uhrhane написал(а):

...

...

one can always move the mouse pointer to top of screen to get back their menu bar. Is that a Mac thing?

Yes, this is how fullscreen applications regularly work on OS X.

...

Mousing around in a fullscreen flash app on Linux or Windows 7 certainly doesn't pop up a menu bar when I hit the top. And the way out is always to hit ESC [although there's often a button as well, depending on the application], so I'm not sure what the problem with fullscreen mouse lock would be.

I do not have recent knowledge of Linux, but my understanding is that Windows UI is undergoing a major redesign with Metro, so basing APIs largely on how Windows used to work up to version 7 may not be future proof.

Yes, but >90% of internet users are not familiar with Metro. They are familiar with Windows as it exists today (XP thru 7). -Darin

- WBR, Alexey Proskuryakov

_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev