[Webkit-unassigned] [Bug 153780] New: Debug test bots exit early frequently. Raise exit-after-n-crashes-or-timeouts to 100.

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=153780

            Bug ID: 153780
           Summary: Debug test bots exit early frequently. Raise
                    exit-after-n-crashes-or-timeouts to 100.
    Classification: Unclassified
           Product: WebKit
           Version: Other
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: clopez at igalia.com
                CC: bugs-noreply at webkitgtk.org, cgarcia at igalia.com,
                    lforschler at apple.com, mcatanzaro at igalia.com

We are having the GTK+ debug test bot abort early frequently https://build.webkit.org/builders/GTK%20Linux%2064-bit%20Debug%20%28Tests%29?numbuilds=200

Despite our best efforts to have it under control, there are still many tests timing out or asserting on the GTK+ debug bot.

The current threshold of 50 timeouts or crashes for abort early looks a bit tight to me.

The last time that this threshold was raised was 3 years ago on http://trac.webkit.org/changeset/150303 (it was raised from 20 to 50).

At that time, we had 34459 tests. Now (as of r195986) we have 44165 tests (28% more).

Currently, not only the GTK+ Debug bot is aborting early. Also 'Apple El Capitan Debug WK1' seems to abort early from time to time.

I see more value on the bot completing the test suite than aborting early. At least for me is more useful to see 85 crashes than to see an abort early because >50 crashes was reached.

So I propose to raise this to 100. I think is a reasonable trade-off.

Patch is coming.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.webkit.org/pipermail/webkit-unassigned/attachments/20160202/e535e2ba/attachment.html>