[Webkit-unassigned] [Bug 263260] New: Move testharness.js tests to using a timeout_multiplier

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=263260

            Bug ID: 263260
           Summary: Move testharness.js tests to using a
                    timeout_multiplier
           Product: WebKit
           Version: WebKit Nightly Build
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: gsnedders at apple.com
                CC: webkit-bug-importer at group.apple.com

Currently we set the internal testharness.js timeout in our testharnessreport.js, totally overriding testharness.js's default behaviours:

  setTimeout(timeout, testRunner.timeout * 0.9);

For Apple ports (though see also bug 263252), this gives for non-slow tests in most configurations a 27s timeout, and for slow tests in most configurations a 135s timeout.

Upstream wptrunner uses a 10s timeout for most tests, or a 60s timeout for "long" (i.e., slow) tests. That said, this is configurable in a bunch of ways; debug Firefox for example uses a 3x multiplier and therefore ends up with 30s/180s timeouts. (Within WebKit, the glib port uses double the timeout for debug, which would mean 20s/120s timeouts, still both shorter than what we currently use.)

In general, we should probably also be setting timeout_multiplier rather than overriding the timeout directly: this allows tests to use test.step_timeout() which scales its timeout based on the multiplier, which hopefully saves us some flakiness.

There's two main approaches we could go with here:

One is to set it based on the current TestRunner timeout and the default WPT timeout (something like testRunner.timeout * 0.9 / wptTimeout would maintain our current behaviour, but would always result in >1.0 multipliers, which is probably not necessary so we might want a smaller constant than 0.9).

The other is to actually change how we deal with timeouts throughout run-webkit-tests (which probably isn't a bad idea in general!) to have a base timeout and then just repeatedly increase the multiplier, and pass both into the test runner.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20231017/6dbfc0e6/attachment-0001.htm>