[Webkit-unassigned] [Bug 132350] AX: [PATCH] procedures for automated atk text and caret tests

Thu May 1 07:10:56 PDT 2014

https://bugs.webkit.org/show_bug.cgi?id=132350

--- Comment #7 from Mario Sanchez Prada <mario at webkit.org>  2014-05-01 07:11:16 PST ---
(In reply to comment #5)
> Ok, I forget about this patch. I wanted to achieve a goal in a way that turned out to be unacceptable.
> 
> Mario, could you help me do it in an acceptable way? I would like to have a function that would receive 2 arguments and perform 3 tests. Details as described in the description of this bug report. If possible, actually only 1 argument could be given, the html source, because perhaps webkit could give us the text using non-accessibility functions.
> 
> Is extending AccessibilityController a way to go? Would you suggest a signature of a function or functions?

I think there's a bit of confusion here, which I'm not surprised about since everything related to a11y in WebKitGTK is quite confusing anyway :)...

As per the description of this bug:

> testatk.c lacks a convenient machinery to quickly add more test. I have an idea how to improve it. I think of a test that would get 2 things on input:
> 
> 1. html source
> 2. corresponding text
> 
> The test procedure would verify 3 things:
> 
> A. Whether all text atk components summed up give the expected text.
> B. Whether caret may be inserted into all text components at all offsets.
> C. Whether caret movements cause text-caret-moved events.
> 

I think you might not be understanding how we test things in WebKit and, more specifically, accessibility related features. Let me try to summarize some points:

We do have 2 kind of tests of interest:

  * Unit tests: used to test APIs, both public (e.g. WebKit2GTK+ API) or private (between internal components).
  * Layout tests: used to test the features of the engine by feeding HTML/CSS/JavaScript content to a WebView and then performing some checks over that using some helper tools (DumpRenderTree for the WebKit1, WebKitTestRunner for WebKit2).

So, testatk.c belongs to the old set of unit tests from WebKit1, because it's just a C file explicitly checking ATK functions which is indeed an API consumed outside of webkit (by the ATSPI-ATK bridge). So, even though it was not a "true WebKit API" as for instance is the case of the WebKit2GTK+ API, it was more or less fine for WebKitGTK+ when based in WebKit1 for the following reasons:

 * It gave complete control on how to test ATK specific functions and signals without bothering other ports
 * It was possible to do it because we only had one process, so we could access the ATK world from the unit tests even if those actually lived in WebCore
 * It was easy and quick :)

However, the testatk.c posed a big number of problems too, which rendered into a not feasible approach when WebKit2 arrived. Some reasons:

 * It does not scale well: not easy to add more cases in an organized way (as you noticed)
 * It forced other ports using ATK for a11y (EFLWebKit) to either not have a11y unit tests or code their own. Duplication is bad
 * In WebKit2GTK+ is simply not possible to use the ATK API since the unit tests are in the UI Process and the ATK world lives in the Web Process. We could use AT-SPI instead, but then we would be getting into an additional layer of trouble, would not be testing exactly the same things and still would have the other problems
 * Probably something else I'm missing now...

So, what we decided to do was to convert the unit tests in testatk.c into regular layout tests that would end up testing exactly the same ATK calls under the hood, by means of the helper tools in place: DumpRenderTree (for WebKit1) and WebkitTestRunner (for WebKit2). That's precisely what Joanmarie was doing during the WebKitGTK+ hackfest in december, as you can see here:

http://trac.webkit.org/changeset/160316
http://trac.webkit.org/changeset/160365

As you can see there, she converted a bunch of unit tests in testatk.c into layout tests, both by (1) writing new HTML files that would check the same things AND (2) adding the needed support -if not present already- in the ATK versions of AccessibilityUIElement in DumpRenderTree and WebKitTestRunner to make sure we could test that by using regular Javascript calls.

So, if you really want to help with this (which is extremely appreciated, since I barely would have time for that these days), I believe a good point to start from would be to convert into layout tests the remaining tests that were left in testatk.c after Joanie's patches, adding support when needed in the accessibility related classes (AccessibilityUIElement, AccessibilityController) for ATK in WebKitTestRunner only (as DRT is gone too from trunk these days). Those files live in Tools/WebKitTestRunner/InjectedBundle/atk, btw.

How does that sound?

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.