[webkit-dev] are fuzzer tests appropriate layout tests?

[Darin Adler]

On Jun 12, 2012, at 5:17 PM, Ojan Vafai <ojan at chromium.org> wrote:

> It's great to use a fuzzer in order to find cases where we're broken and then make reduced layout tests from those.


Generally we do require a test each time we fix a bug. So it’s a strategy for the project to always make reduced tests when we find a bug.

But using a fuzzer to find bugs and then making a regression test for each bug we find will not give us great coverage. We’d like tests that cover lots of the code paths in WebKit, even the ones without bugs.

I’m not saying we should necessarily keep fuzzer-style tests, but to replace them we would need to add tests with good coverage, going beyond regression tests for bugs that existed in the project at one point.

At one point, I remember Geoff Garen encouraging a fuzzer-type approach to making some tests for an SVG path parser, as an alternative to my plan of making a test that covered every branch in the parser code. I don’t remember what we ended up doing in that case.

-- Darin