[webkit-dev] are fuzzer tests appropriate layout tests?
mitz at apple.com
Wed Jun 13 09:08:07 PDT 2012
On Jun 12, 2012, at 5:17 PM, Ojan Vafai <ojan at chromium.org> wrote:
> See https://bugs.webkit.org/show_bug.cgi?id=87772.
> It's great to use a fuzzer in order to find cases where we're broken and then make reduced layout tests from those. The viewspec-parser tests are themselves just a fuzzer though. Granted, they are deterministic by avoiding using an actual random function, but I don't think throwing randomly generated bits at a parser is appropriate for layout testing. If nothing else it's very slow.
> These tests regularly timeout on the Chromium debug bots and occasionally timeout on the Apple Lion bots. Even on the bots where they don't timeout, they're slow. I don't it makes sense to spend 1+ minutes running these 5 tests when more targeted reductions could get the same effective coverage much faster.
> Am I wrong? If not, does anyone object to moving these tests over to ManualTests or just deleting them entirely?
I am not familiar with the viewspec-parser tests and their history, but I agree in principle that fuzzers and their raw output rarely make for the best way to regression test.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the webkit-dev