[Webkit-unassigned] [Bug 196505] New: [ Mac ] Layout Test accessibility/mac/press-not-work-for-disabled-menu-list.html is a flaky failure

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=196505

            Bug ID: 196505
           Summary: [ Mac ] Layout Test
                    accessibility/mac/press-not-work-for-disabled-menu-lis
                    t.html is a flaky failure
           Product: WebKit
           Version: WebKit Nightly Build
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: sroberts at apple.com
                CC: lforschler at apple.com

The following layout test is flaky on Mac WK1, WK2 Release and Debug

accessibility/mac/press-not-work-for-disabled-menu-list.html

Probable cause:

Test was added in https://trac.webkit.org/changeset/241190/webkit and is a flaky failure on its entire available dashboard history for Mojave WK1, but through local testing I am able to get it to fail on Release and Debug WK2 as well, just not as consistently. 

Local testing for WK1 yields approximately 20 failures in 500 iterations. WK2 testing yields about 2 failures in 500 iterations. 

Reproduced with:

run-webkit-tests accessibility/mac/press-not-work-for-disabled-menu-list.html --iterations 500 -f -1

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=accessibility%2Fmac%2Fpress-not-work-for-disabled-menu-list.html

Diff:

--- /Volumes/Data/slave/mojave-release-tests-wk1/build/layout-test-results/accessibility/mac/press-not-work-for-disabled-menu-list-expected.txt
+++ /Volumes/Data/slave/mojave-release-tests-wk1/build/layout-test-results/accessibility/mac/press-not-work-for-disabled-menu-list-actual.txt
@@ -1,3 +1,4 @@
+CONSOLE MESSAGE: line 56: TypeError: undefined is not an object (evaluating 'menulist.addNotificationListener')

 This tests that menu lists that are disabled will not be triggered by AXPress actions.

@@ -7,7 +8,7 @@
 Got notification: AXPressDidFail
 Got notification: AXPressDidFail
 Got notification: AXPressDidFail
-PASS successfullyParsed is true
+FAIL successfullyParsed should be true (of type boolean). Was undefined (of type undefined).

 TEST COMPLETE

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20190402/7bb2c7c6/attachment.html>