[webkit-dev] DRT/WTR should clear the cache at the beginning of each test?

Dirk Pranke dpranke at chromium.org
Wed Aug 8 14:35:08 PDT 2012

On Wed, Aug 8, 2012 at 10:47 AM, Ojan Vafai <ojan at chromium.org> wrote:
> See https://bugs.webkit.org/show_bug.cgi?id=93195.
> media/W3C/video/networkState/networkState_during_progress.html and
> media/video-poster-blocked-by-willsendrequest.html are flaky on all
> platforms because they behave differently if the loaded resource is cached.
> Every time I've taken a stab at reducing test flakiness, I've come across at
> least a few tests that pass when run as part of the test suite, but fail
> when run by themselves (or in parallel) because they accidentally expect an
> image or something to be in the cache.
> I think it would make the tests more maintainable if we cleared the cache
> before each test run. This is *not* before each page load though. So tests
> that do multiple page loads will still test cross-navigation caching
> behavior.
> While it's true that we could one-off fix each of these tests, it's usually
> very time consuming to figure out that caching is the problem, that's
> assuming anyone takes the time to look into why the test is flaky in the
> first place.
> Any objections?

Given that the way we run tests in parallel in NRWT means that
different processes get different lists of tests each time, it sounds
like we may be getting a fair amount of nondeterminism from the cache
not being cleared between tests. That seems bad, so I'm in favor of
clearing the cache :)

-- Dirk

More information about the webkit-dev mailing list