[Webkit-unassigned] [Bug 132280] [GTK] Lots of accessibility tests are flaky

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=132280





--- Comment #5 from Mario Sanchez Prada <mario at webkit.org>  2014-05-29 03:21:37 PST ---
(In reply to comment #4)
> (In reply to comment #3)
> > I've checked the results over time using Carlos' WebKit test hunter[1], and I can't see any of these tests failing since 3 weeks ago (last failure was with [2]), which I'm not sure can be considered as "flaky enough" to put them in the TestExpectations file.
> > 
> > Calvaris, what do you think?
> > 
> > [1] https://github.com/clopez/webkit-testhunter
> > [2] http://trac.webkit.org/changeset/168469
> 
> By using the script I still see some text failures not so long ago so they still fail, though you consider those do not happen ofter enough.

Good point. I've just found the text diff for one of those tests (platform/gtk/accessibility/text-at-offset-newlines.html) in r168469 and this is what I saw: 

--- /home/slave/webkitgtk/gtk-linux-64-release/build/layout-test-results/platform/gtk/accessibility/text-at-offset-newlines-expected.txt
+++ /home/slave/webkitgtk/gtk-linux-64-release/build/layout-test-results/platform/gtk/accessibility/text-at-offset-newlines-actual.txt
@@ -1,3 +1,24 @@
+Accessibility object emitted "state-change:busy = 1" / Name: "(No name)" / Role: 93
+Accessibility object emitted "children-changed:add = -1" / Name: "(No name)" / Role: 48
+Accessibility object emitted "property-change:accessible-parent" / Name: "(No name)" / Role: 48
+Accessibility object emitted "property-change:accessible-parent" / Name: "(No name)" / Role: 48
+Accessibility object emitted "property-change:accessible-parent" / Name: "(No name)" / Role: 48
+Accessibility object emitted "property-change:accessible-parent" / Name: "(No name)" / Role: 48
+Accessibility object emitted "property-change:accessible-parent" / Name: "(No name)" / Role: 48
+Accessibility object emitted "property-change:accessible-parent" / Name: "(No name)" / Role: 48
+Accessibility object emitted "property-change:accessible-parent" / Name: "(No name)" / Role: 48
+Accessibility object emitted "property-change:accessible-parent" / Name: "(No name)" / Role: 48
+Accessibility object emitted "property-change:accessible-parent" / Name: "(No name)" / Role: 48
+Accessibility object emitted "property-change:accessible-parent" / Name: "(No name)" / Role: 48
+Accessibility object emitted "property-change:accessible-parent" / Name: "(No name)" / Role: 48
+Accessibility object emitted "property-change:accessible-parent" / Name: "(No name)" / Role: 48
+Accessibility object emitted "property-change:accessible-parent" / Name: "(No name)" / Role: 48
+Accessibility object emitted "children-changed:add = -1" / Name: "(No name)" / Role: 93
+Accessibility object emitted "children-changed:add = -1" / Name: "(No name)" / Role: 93
+Accessibility object emitted "children-changed:add = -1" / Name: "(No name)" / Role: 93
+Accessibility object emitted "children-changed:add = -1" / Name: "(No name)" / Role: 93
+Accessibility object emitted "children-changed:add = -1" / Name: "(No name)" / Role: 93
+Accessibility object emitted "property-change:accessible-parent" / Name: "(No name)" / Role: 71
 This is a test. This is the second sentence. And this the third.

 This tests the ability to get element text for different granularities and offsets.


This is very weird. For some reason, the test is enabling "logging accessibility events" (as if accessibilityController.logAccessibilityEvents()) was called for those tests, which is not), and that is causing that all that text is being dumped "accidentally", therefore causing the text failure. This is *clearly* the cause of that problem.

Now, to be honest, I'm not sure why that thing happens, although it sounds like related to the setup/teardown processes that happens for every single test, which seems indeed to be flaky, at least in what it relates to logAccessibilityEvents() which now I reckon that has not been a very good idea right from the start :/, as it has presented many problems and, on top of that, it should not be needed if we had addNotificationListener() implemented, which we do have now.

Also, the only tests actually using logAccessibilityEvents() these days in GTK/EFL are this ones: 

  platform/gtk/accessibility/aria-slider-required-attributes.html
  platform/gtk/accessibility/combo-box-collapsed-selection-changed.html

So, all in all, perhaps it's better to adapt those tests not to use logAccessibilityEvents() at all, and replace the implementation in WKTR with a dummy one, so it does not cause more harm anymore (we can't simply remove it, since logAccessibilityEvents() is implemented in the win port).

I'll work on this change soon, probably tomorrow (as today I need to leave early due to personal matters)

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.