[webkit-dev] Test flakiness.

[Samuel White]

Thanks for the tips Alexey. I suspected order was to blame and wrote a script to run all the LayoutTests/platform/mac/accessibility tests as tuples with the flaky test coming second. The winning combination that reproduces the flaky failure for me:

Tools/Scripts/run-webkit-tests --debug LayoutTests/platform/mac/accessibility/document-links.html LayoutTests/platform/mac/accessibility/search-predicate-element-count.html

Odd/interesting that order matters. Still chasing down the details, thanks again.

On Dec 3, 2013, at 3:23 PM, Alexey Proskuryakov <ap at webkit.org> wrote:

> 
> 03 дек. 2013 г., в 14:11, Tim Horton <timothy_horton at apple.com> написал(а):
> 
>> It would be ideal if you could figure out *why* the test is flaky. What state are the tests that run before it setting that isn’t getting reset? Can you narrow it down to a pair of tests?
> 
> One way to do this is to run accessibility tests with "--child-processes=1 --verbose" options, and to start bisecting if that makes the issue reliable.
> 
> Another useful approach is to make the tests run multiple times with "--child-processes=1 --verbose —repeat-each=10".
> 
> - WBR, Alexey Proskuryakov
> 
>