[webkit-dev] Proposal: Rect based HitTest for a better touch experience

[David Hyatt]

On Jun 2, 2010, at 2:46 PM, Antonio Gomes (:tonikitoo) wrote:

> Hi,
> 
> As most of you have experienced, the precision of a mouse click on
> Desktop applications, including Web browsers, is much higher than the
> precision of a touch tap on a mobile device. It is not a new problem,
> and many solutions have been proposed to improve that situation. One
> of the common solutions is to make a small target’s tap-sensitive area
> larger, invisibly, than the visible target itself. Of course, it would
> not work on a mobile Web browser, since it'd end up breaking the
> desired layout of any non-simple Web site.
> 
> Mozilla has addressed this problem by implementing the SmartTap
> feature [1]. The solution is simple: a "nodesFromRect" method is
> available for applications {fennec, firefox, etc}. As the name
> implies, given a rect (corresponding to tapped area), it returns a
> list of candidate "mouse clickable" target nodes for the tap. This
> list is sorted by z-order, larger intersection area, most visited (in
> case of links), etc. Call sites can then pick the first element on the
> list, or use any different heuristic to choose the target.
> 
> [1] https://bugzilla.mozilla.org/show_bug.cgi?id=489127
> 
> After some initial research, mostly understanding the HitTest system
> of WebCore, I am sure we can provide something similar as a cross
> platform solution for the problem.
> 
> For the moment, my plan is to use better hit testing to make it easier
> for users to click & focus hyperlinks and input fields. For that, the
> HitTest system would have to be modified to receive a Rect as input,
> instead of a Point. It would also have to hold a list of possible
> target candidate nodes, instead of only a single target node as it
> does currently. In deep, the many ::nodeAtPoint implementations would
> have to be adjusted to work with a Rect, not a Point, and probably
> renamed to something similar to nodesAtRect. Much of the current
> HitTest logic in RenderLayer would not need changes, though.
> 
> To keep the current mouse behavior (which is clearly point-based), we
> could just pass rects with width and height equal to 1, so a Rect(x,
> y, 1, 1) would behavior equally or very similarly to Point(x, y).
> 
> Before going this way, I would like to know of any known problem or
> show stoppers.

I really don't think hit testing needs to be modified to get what you want.  You can do a scattershot sampling using multiple candidate points within the rect and apply whatever heuristics you want to choose a node.  I'm also not convinced that we should be complicating core hit testing code in this way, since I suspect this is one of those areas where mobile platforms will each want to do their own slightly different thing anyway.

dave
(hyatt at apple.com)