[webkit-dev] are fuzzer tests appropriate layout tests?

[Dirk Pranke]

On Wed, Jun 13, 2012 at 12:05 PM, Darin Adler <darin at apple.com> wrote:
> On Jun 12, 2012, at 5:17 PM, Ojan Vafai <ojan at chromium.org> wrote:
>
>> It's great to use a fuzzer in order to find cases where we're broken and then make reduced layout tests from those.
>
>
> Generally we do require a test each time we fix a bug. So it’s a strategy for the project to always make reduced tests when we find a bug.
>
> But using a fuzzer to find bugs and then making a regression test for each bug we find will not give us great coverage. We’d like tests that cover lots of the code paths in WebKit, even the ones without bugs.
>
> I’m not saying we should necessarily keep fuzzer-style tests, but to replace them we would need to add tests with good coverage, going beyond regression tests for bugs that existed in the project at one point.
>

I have always been under the impression that  LayoutTests were not
just intended for preventing regressions to bugfixes, but that we
should also be adding tests to establish correctness (and hopefully
achieve good coverage) there. Is that not the case?

Certainly I would expect the imported test suites to be testing
correctness and not just be regression bug fixes.

-- Dirk