[webkit-dev] DRT/WTR should clear the cache at the beginning of each test?
fischman at chromium.org
Fri Oct 26 13:55:35 PDT 2012
Should we add random sleeps to DRT? It'll certainly help find some
regressions (and even security bugs).
Of course the down-side is that it makes tests non-repeatable and difficult
to reason about.
I'm baffled by your priorities and don't know how to continue this
conversation productively. Sorry.
On Fri, Oct 26, 2012 at 12:43 PM, Alexey Proskuryakov <ap at webkit.org> wrote:
> 26.10.2012, в 11:04, Antti Koivisto <koivisto at iki.fi> написал(а):
> The reality is that this "test coverage" today shows up as flakiness and
>> so is ignored anyway, meaning we don't actually have useful coverage here.
>> Even when flakiness is investigated, the "fix" is to cache-bust using
>> unique URL params, which just means we "lose" the coverage you describe for
>> that test, anyway.
> I think that this is the real issue here. Test flakiness is very important
> to investigate, this often leads to discovery of bad bugs, including
> security ones. The phrase "flaky test" often misplaces the blame.
> When making cache related changes I have frequently found bugs from my
> patches because some seemingly random test started failing and I
> investigated. Without the test coverage some of those bugs would probably
> now be in the tree.
> I agree with Antti. Finding regressions is what tests are for, and it
> would be difficult to make enough explicit tests to compensate for such
> loss of coverage. It would certainly be very unfortunate to lose test
> coverage without even an attempt to compensate for that.
> - WBR, Alexey Proskuryakov
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the webkit-dev