[webkit-dev] Accessibility Object Searching

[Samuel White]

On Jun 22, 2011, at 3:21 PM, Dominic Mazzoni wrote:

> On Wed, Jun 22, 2011 at 3:14 PM, Samuel White <samuel_white at apple.com> wrote:
>> Thanks for the feedback. I've been prototyping this idea here and it does included text searching as you mentioned. My philosophy was to put the search functionality right into the AccessibilityObject class and leave the job of exposing external APIs up to the experts of each platform save for Mac. In the Mac accessibility code I have this functionality exposed under accessibilityAttributeValue:forParameter: method where the attribute one must pass is AXSearchPredicate.
> 
> I'd love to see the actual proposed spec for AXSearchPredicate. Right
> now your proposed change is all in terms of AccessibilityObject, but
> that's very WebKit-specific. If you're proposing a new capability to
> be added to NSAccessibility, I think it's important to talk about what
> the spec means at that level, so that we're sure it could be
> implemented by another, non-WebKit browser.

The AXSearchPredicate attribute is a working concept at the moment. The parameter is an NSDictionary that contains the following:

An AccessibilityObject to start searching from.
An NSString that specifies the type of search (ex: @"AXLinkSearchKey").
An NSString that specifies the search direction (ex: @"AXDirectionNext").
An NSString that holds the text to search for.

> It might be nice to propose it to, say, IAccessible2 - I know they're
> already in the middle of considering proposals for the next version,
> so this might be a great place to get feedback.
> 
> https://wiki.mozilla.org/Accessibility/IA2_1.3
> 
> I'd be happy to try to provide an intro if you'd be willing to
> entertain discussion at least.

The NSAccessibility API additions are designed to meet the needs of Mac accessibility clients. We're providing in webkit a generic method so that any platform API can take advantage of it. I'll CC you on the patch so you can look at the generic mechanism in order to make sure it meets the needs of another platform's API.

>> As for implementation details, I'd like to outline how a search is performed in my prototype for you. As I stated above, searching happens in the AccessibilityObject class. The function doing the heavy lifting takes an AccessibiltySearchPredicate which is just a simple data structure that holds the following:
>> 
>> An AccessibilityObject to search within.
>> An AccessibilityObject to start searching from (ex: search for X within object A but after object B).
>> A direction to search in.
>> A search type (link search, heading search, bold text search, etc).
> 
> Could you clarify this one? Are you going to enumerate possible
> searches? What about just searching for a particular role, would that
> work?

The Search type exists because I don't think there is a clear relation to just axRoles. For example, if you are searching for the next element containing bold text you would need font information (italic text, underlined text, and italic text come to mind as well). Since a key is used we don't need to enumerate but can instead jump to the correct case for a given search.

>> Search text to look for.
>> Number of AccessibilityObjects to return.
> 
> These sound fine.
> 
> - Dominic
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20110622/54eaea48/attachment.html>