[webkit-dev] pointer events specification - first editors draft

[Rick Byers]

>> The premise of the specification is that using mouse event and touch
events
>> interchangeably is needed. In reality, nobody was really asking for that
>> because it is a terrible idea. You can already easily unify pen, touch
and
>> mouse by writing very little JavaScript, but it is only useful in very
few
>> cases.

It's not accurate to say "nobody" is asking for it.   I agree that you
generally need to design explicitly for touch and mouse interactions, and
it's a legitimate concern that having a unified model may encourage
developers to ignore important differences.  However, developers are
already attempting to unify the events today when the right design involves
treating single-finger touch actions the same as mouse actions.  For
example, Google maps transforms both touch events and mouse events to a
common unified event system, as does the PlayN game engine (
http://code.google.com/p/playn/). There has been at least SOME positive
response in the web developer community to the idea of a unified event
model, so I don't think it's fair to dismiss this idea as having no value
out of hand.

> >> You can already easily unify pen, touch and
> >> mouse by writing very little JavaScript, but it is only useful in very
few
> >> cases.

It's not true that you can correctly unify the two with "very little
JavaScript".  In particular, it's a subtle but significant difference that
touch events are always delivered to the node that was first touched, while
mouse events are delivered to whatever node the mouse cursor is currently
over.  The similar-but-different model is dangerously deceptive.  For
example, I recently came across an issue with a scrollable list library
where occasionally touch scrolling would just stop working.  We eventually
realized that the library would sometimes remove nodes that had scrolled
outside the visible region.  So when it removes a node which just happened
to be the one under the finger when scrolling started, all the touchmove
events were no longer being delivered to the container (with not even a
touchcancel).  Removing nodes while their container is being manipulated is
perfectly reasonable for the mouse model, but is subtly difficult with
touch events.  The only libraries I've seen that try to unify the
targetting behavior of mouse and touch events are doing it by emulating
pointer events eg. https://github.com/toolkitchen/PointerEvents.

>> In particular, having apps state clearly and declaratively [3] whether a
touch that starts on an
>> element should result in browser behavior like scrolling or events which
can be handled from
>> JavaScript is potentially a big win for touch scrolling performance
(something we struggle with in Chrome).
>
> This is a straw man.
> Chromium has implementation issues there, adding a new spec is not the
solution to your problems.
>
> Having touch-action == "auto", the default, will have the exact same
issues as you have today.
> Having touch-action == "none" does not buy you much since it would have
been implemented
> today by responding to the very first event.

This is incorrect.  Let me ellaborate on the benefit here.  The fundamental
difference between the two models is that touch events imply (although it's
not well specified, and implementations differ) that it's possible to
receive touchmove events while an element is being scrolled.  In such a
case, we can't really do threaded scrolling (especially if calling
preventDefault on the touchmove is supposed to stop scrolling at that
point).  This makes it incredibly easy for web developers to accidentally
defeat fast scrolling optimizations (just by having a touchmove handler on
an element which happens to contain a scrollable div).  For example,
http://store.apple.com/us has a touchmove handler on the document (for no
apparent good reason that I can see), which means that (in the chrome
desktop implementation at least), we're unable to do scrolling without
blocking on the main webkit thread because the page might really be
expecting to receive events for the touch.  In rare scenarios (when an app
needs to receive touchmove events for a parent element, but one if it's
children is scrollable) threaded scrolling appears impossible without
potentially breaking changes to the touchevent model.

Pointer events, on the other hand, were designed from the start to avoid
this issue.  They require the developer to explicitly state whether they
want scrolling OR events.  The spec is explicit that you will NOT receive
events when the browser is performing a default action like scrolling.  So
"touch-action: auto" means that once a scroll gesture is detected (even
when the element itself isn't actually scrollable), you get a pointercancel
event indicating that the page will no longer receive events for the touch.
 If the developer REALLY wants to get all the events, they have to
explicitly state that they're disabling scrolling on the element with
touch-action: none.   IMHO, it's far better to require developers to make
an explicit trade-off between scrolling and app-level events, than it is to
let them believe they can have it both ways without appreciating the
performance implications.  I question whether a modern touch API should
even give developers the option of having janky scrolling, nevermind
whether it should be so easy to opt-in to that mode accidentally.

Do you agree there is some room for improvement in the touch event model
here?  If so, what path specifically do you propose for getting such
improvements into an open standard and into WebKit?

Note that it's no longer just interoperability with IE that we're concerned
about - Mozilla and Opera have both also indicated that they see pointer
events as the future, and that no new work should be done on touch events.
 What options do we have for moving touch input forward on the open web?

Rick

On Sat, Dec 1, 2012 at 3:17 AM, Maciej Stachowiak <mjs at apple.com> wrote:

>
>
> On Nov 28, 2012, at 8:23 AM, Adam Barth <abarth at webkit.org> wrote:
>
> > It looks like this thread got dropped over Thanksgiving.  (Responses
> inline.)
> >
> > On Thu, Nov 22, 2012 at 1:43 PM, Benjamin Poulain <benjamin at webkit.org>
> wrote:
> >> On Wed, Nov 21, 2012 at 2:42 PM, Rick Byers <rbyers at chromium.org>
> wrote:
> >>> Outside of the design details, are there any thoughts or concerns
> >>> about WebKit getting an implementation of pointer events at some
> >>> point?
> >>
> >> In my opinion, the pointer events spec is a bad idea (at least, in its
> >> current state). I think adding it to WebKit would be hurting both the
> Web
> >> and WebKit.
> >>
> >> The premise of the specification is that using mouse event and touch
> events
> >> interchangeably is needed. In reality, nobody was really asking for that
> >> because it is a terrible idea. You can already easily unify pen, touch
> and
> >> mouse by writing very little JavaScript, but it is only useful in very
> few
> >> cases.
> >>
> >> For the common case, where you want to use touch and mouse events
> >> differently, the pointer events makes it harder to write correct code.
> Each
> >> event have a set of attributes with different meaning depending on the
> event
> >> type.
>
> I agree with Benjamin's technical concerns about this feature. Combining
> the very different models of touch and mouse is a bad idea. They are
> distinct and for anything beyond the trivial, you really need to treat them
> separately to get a good outcome. The basic premise of pointer events seems
> to be over-abstraction at the expense of the user experience.
>
> >
> > I'm sympathetic to these concerns, but, unfortunately, I don't see any
> > other path that leads to interoperability with Internet Explorer.
>
> In principle, other things being equal, interop with IE would be better
> than not having it. But other things are not equal. Their proposal is
> technically unsound.
>
> Further, I do not much of a pragmatic interop concern in this case. IE is
> so far not very relevant on device classes that generally feature touch
> UI[*]:
> <
> http://marketshare.hitslink.com/browser-market-share.aspx?qprid=0&qpcustomd=1>.
> The trajectory of Microsoft's touch-oriented OS efforts does not seem
> likely to change this trend in the near future.
>
> Clearly Microsoft has looked at this data and decided interop with other
> browsers in this area is not very important to them, relative to other
> goals. That's so even with a significant disadvantage in market share and
> deployed content in the relevant addressable market. Given that, why should
> IE's interoperability be more important to us than it apparently is to
> Microsoft?
>
> In conclusion, I do not think interop with IE is more important here than
> technically correct design. I'd say interop trumps sound design only when
> there is a mass of deployed content backing up the interoperability need.
>
> Regards,
> Maciej
>
>
> [*] I know that Windows for desktop/laptop machines also supports
> touchscreens, and some hardware that's primarily meant for
> keyboard-and-mouse type traditional input also features a touchscreen as a
> potential extra source of input. But from what I can discern, this is more
> a neat gimmick than something that is truly compelling to users or
> something that Web developers are rushing to take advantage of. Content
> that is specialized for touch UIs is overwhelmingly authors for Android
> and/or iOS. If I'm wrong on that, then I'd like to see some actual data to
> the contrary.
>
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo/webkit-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20121204/f9ef4f0a/attachment.html>