[webkit-dev] [webkit-changes] [52980] trunk/LayoutTests

[Alexey Proskuryakov]

On 08.01.2010, at 11:56, Eric Seidel wrote:

>> I think that having this particular test enabled is much more  
>> important than
>> having the patch it was affecting landed.
>
> Agreed.  But, I think it's inexcusable for a test to affect later
> results.  I skipped it more on those grounds than that the test it was
> enabling was important.


It's not clear where the bug is though. If there is an uninitialized  
memory read in a later test, then earlier tests that randomly affect  
it can be perfectly correct. The fact that on other platforms the  
problem was not fixed support this hypothesis to some degree.

The opposite is sometimes true, as well - there were whole series of  
randomly crashing tests added to Skipped list by different persons,  
even though the fault lied with an earlier test.

As I said before, it's good when flaky tests cause pain and impede  
progress (as long as those tests are correct, of course). That's their  
way of making us investigate problems which can quickly turn into  
exploitable security bugs in shipping software otherwise.

- WBR, Alexey Proskuryakov