[webkit-dev] why these tests are still marked Failure? r148002

[Mario Sanchez Prada]

Hi Jarek,

> -----Original Message-----
> From: webkit-dev-bounces at lists.webkit.org [mailto:webkit-dev-
> bounces at lists.webkit.org] On Behalf Of Jarek Czekalski
> Sent: 03 May 2014 08:35
> To: webkit-dev at lists.webkit.org
> Subject: [webkit-dev] why these tests are still marked Failure? r148002
> 
> r148002 added Failure for the test
> platform/gtk/accessibility/combo-box-collapsed-selection-changed.html
> 
> Then there was bug #114259 [1], which was resolved and fixed a year
> ago.
> 
> Shouldn't the tests be turned into Pass again?

Actually, the tests are still failing, so that's why they are marked as
failure:

http://build.webkit.org/results/GTK%20Linux%2064-bit%20Release/r168352%20%28
47152%29/platform/gtk/accessibility/combo-box-collapsed-selection-changed-pr
etty-diff.html
http://build.webkit.org/results/GTK%20Linux%2064-bit%20Debug/r168352%20%2841
926%29/platform/gtk/accessibility/aria-slider-required-attributes-pretty-dif
f.html

> On my box this test fails, but I'm confused about this situation. I am
> currently looking at logAccessibilityEvents method, but all the gtk
> tests using this method are marked failure. Help me please.

In the GTK port, logAccessibility events is something we used in the past to
test that the right port specific (e.g. ATK) signals were being emitted when
running certain layout tests, so that we could actually look for them in the
generated output.

However, over the time (last year) we shifted into mapping those signals in
DRT/WKTR and using AccessibilityUIElement::addEventListener() and
AccessibilityController::addNotificationListener() from the layout tests to
look for those events happening, which is a much more accurate (and less
prone to flakyness) way of checking ATK signals.

Unfortunately, those tests you just found were not migrated from
logAccessibilityEvents() to the other mechanism, and kept in the
TestExpectations file as the current output is not exactly the one that was
expected some time ago.

So, the right thing to do here to really fix those tests would be to migrate
them to using addNotificationListener() / addEventListener() and stop using
logAccessibilityEvents().

Unfortunately, I think we can't remove logAccessibilityEvents() from WKTR
since I believe there are other ports that are using it (probably in a more
reliable way).

Hope this shows some light, and sorry for the mess (which is partly my
fault)

Thanks,
Mario