[webkit-dev] Coming Soon: new-run-webkit-tests

[Brady Eidson]

On Apr 9, 2010, at 4:41 PM, Alexey Proskuryakov wrote:

> 
> On 09.04.2010, at 16:32, Brady Eidson wrote:
> 
>>> - Exposes more flaky tests due to running tests in a non-deterministic order.
>> 
>> This sounds like a pro to me.
> 
> 
> Existing run-webkit-tests can also do that, via a non-default option. Unreproducible results with default options do sound like a con.

Don't get me wrong - if a test fails reliably due to a specific run order, then the tool needs to be able to record the run-order so the failure can be explored.

But finding more flakey tests by mixing up the run order is - in principle - a good thing.

I'd forgotten that the current tool can mixup the run-order.

We should make the new tool record the order so order-specific failures can be explored.

~Brady

> - WBR, Alexey Proskuryakov
> 
>