[Webkit-unassigned] [Bug 37738] new-run-webkit-tests low timeout is too sensitive to runaway processes or test crashes

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=37738





--- Comment #7 from Ojan Vafai <ojan at chromium.org>  2010-04-19 07:35:56 PST ---
(In reply to comment #6)
> (In reply to comment #5)
> > Sounds like a neat idea. It would somewhat reduce the incentive to make your
> > tests fast though. Maybe there could be a "soft" timeout that would result in a
> > non-failure warning, to make slow tests more visible?

This seems like a great idea. If we could take it one step further and show the
number of slow tests on the waterfall, that might provide incentive to fix
them. What I like about this is that we can be more flexible about the timeout.
The current 6 second timeout (12 seconds on debug) is just what happened to
seem reasonable given the data from the Chromium bots. 

We can start with something conservative (6 seconds?) and make it more
aggressive as we fix the slower tests.

> All timeouts are currently "soft" in that all failures are re-run at the end. 
> Tests which timed out once due to machine load are unlikely to time out twice.

This is half true. If ReportCrash is the problem and your patch causes crashes,
then crashes will happen when you retry and tests might timeout if you don't
have sufficient extra cores on your machine.

> > > Also, is there a way we can kill ReportCrash from NRWT? When I run the tests
> > > locally, I frequently will "sudo killall ReportCrash" in a loop. I'd much
> > > rather have NRWT do that for me. The sudo is the problem of course.
> > 
> > I don't think that would be a good thing to do. The crash logs are very useful
> > for diagnosing what went wrong, especially on the buildbots.
> 
> I've long wanted such an option to run-webkit-tests.  It's easy enough to do.
> Just requires adding a flag to DumpRenderTree to have it catch the necessary
> signals and exit(1) instead of letting the OS catch them for us and calling
> ReportCrash.  I don't think we'd want this mode on by default, but it would be
> useful.

Filed bug 37797.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.