[webkit-dev] Proposal: Rect based HitTest for a better touch experience

[Antonio Gomes (:tonikitoo)]

Hi David.

Although I understand your concern, however I still do not think the
change would complicate things that much, but basically extend the
current support.

For example, in

bool InlineTextBox::nodeAtPoint(const HitTestRequest&, HitTestResult&
result, int x, int y, int tx, int ty)
{
(...)
    IntRect rect(tx + m_x, ty + m_y, m_width, height());
    if (m_truncation != cFullTruncation && visibleToHitTesting() &&
rect.contains(x, y)) {
        renderer()->updateHitTestResult(result, IntPoint(x - tx, y - ty));
(...)
}

"rect.contains(x, y)" would have to be "rect.intersects(myRect)" and
things like that. (of course, not only that).

To simplify stuff, any sorting mechanism I mentioned on my previous
email could be left to the callers, and so. If a port do not want to
take advantage of the new support, the current functionality should
and would be kept, i.e. Rect(x,y,1,1,) would do the trick.

I can certainly prototype something to see the idea in practice before
any final judgment, if you prefer.

Cheers,

On Wed, Jun 2, 2010 at 5:22 PM, David Hyatt <hyatt at apple.com> wrote:
> On Jun 2, 2010, at 2:46 PM, Antonio Gomes (:tonikitoo) wrote:
>
>> Hi,
>>
>> As most of you have experienced, the precision of a mouse click on
>> Desktop applications, including Web browsers, is much higher than the
>> precision of a touch tap on a mobile device. It is not a new problem,
>> and many solutions have been proposed to improve that situation. One
>> of the common solutions is to make a small target’s tap-sensitive area
>> larger, invisibly, than the visible target itself. Of course, it would
>> not work on a mobile Web browser, since it'd end up breaking the
>> desired layout of any non-simple Web site.
>>
>> Mozilla has addressed this problem by implementing the SmartTap
>> feature [1]. The solution is simple: a "nodesFromRect" method is
>> available for applications {fennec, firefox, etc}. As the name
>> implies, given a rect (corresponding to tapped area), it returns a
>> list of candidate "mouse clickable" target nodes for the tap. This
>> list is sorted by z-order, larger intersection area, most visited (in
>> case of links), etc. Call sites can then pick the first element on the
>> list, or use any different heuristic to choose the target.
>>
>> [1] https://bugzilla.mozilla.org/show_bug.cgi?id=489127
>>
>> After some initial research, mostly understanding the HitTest system
>> of WebCore, I am sure we can provide something similar as a cross
>> platform solution for the problem.
>>
>> For the moment, my plan is to use better hit testing to make it easier
>> for users to click & focus hyperlinks and input fields. For that, the
>> HitTest system would have to be modified to receive a Rect as input,
>> instead of a Point. It would also have to hold a list of possible
>> target candidate nodes, instead of only a single target node as it
>> does currently. In deep, the many ::nodeAtPoint implementations would
>> have to be adjusted to work with a Rect, not a Point, and probably
>> renamed to something similar to nodesAtRect. Much of the current
>> HitTest logic in RenderLayer would not need changes, though.
>>
>> To keep the current mouse behavior (which is clearly point-based), we
>> could just pass rects with width and height equal to 1, so a Rect(x,
>> y, 1, 1) would behavior equally or very similarly to Point(x, y).
>>
>> Before going this way, I would like to know of any known problem or
>> show stoppers.
>
> I really don't think hit testing needs to be modified to get what you want.  You can do a scattershot sampling using multiple candidate points within the rect and apply whatever heuristics you want to choose a node.  I'm also not convinced that we should be complicating core hit testing code in this way, since I suspect this is one of those areas where mobile platforms will each want to do their own slightly different thing anyway.
>
> dave
> (hyatt at apple.com)
>
>



-- 
--Antonio Gomes