[webkit-dev] Test flakiness.
samuel_white at apple.com
Tue Dec 3 15:57:27 PST 2013
Thanks for the tips Alexey. I suspected order was to blame and wrote a script to run all the LayoutTests/platform/mac/accessibility tests as tuples with the flaky test coming second. The winning combination that reproduces the flaky failure for me:
Tools/Scripts/run-webkit-tests --debug LayoutTests/platform/mac/accessibility/document-links.html LayoutTests/platform/mac/accessibility/search-predicate-element-count.html
Odd/interesting that order matters. Still chasing down the details, thanks again.
On Dec 3, 2013, at 3:23 PM, Alexey Proskuryakov <ap at webkit.org> wrote:
> 03 дек. 2013 г., в 14:11, Tim Horton <timothy_horton at apple.com> написал(а):
>> It would be ideal if you could figure out *why* the test is flaky. What state are the tests that run before it setting that isn’t getting reset? Can you narrow it down to a pair of tests?
> One way to do this is to run accessibility tests with "--child-processes=1 --verbose" options, and to start bisecting if that makes the issue reliable.
> Another useful approach is to make the tests run multiple times with "--child-processes=1 --verbose —repeat-each=10".
> - WBR, Alexey Proskuryakov
More information about the webkit-dev