[Webkit-unassigned] [Bug 185918] test262/Runner.pm: randomize tests for performance

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=185918

--- Comment #5 from Leo Balter <leonardo.balter at gmail.com> ---
Darin, I'm taking over this patch work as Valerie is in a vacation time this week.

I'm not show how we randomize and keep it for consecutive runs. The results are sorted anyway.

> tests can make problems with one test affecting another into hard-to-reproduce flakiness

Every test run is isolated from each other. The only thing that could be a problem in this case is memory usage from each call to JSC (for each test). This not prevented from the current alternative - tests order as it is by their file path - anyway.

> And we’d like to avoid that even if it’s unlikely.

The only guaranteed way to prevent any conflict here is to run the tests in a single process queue. That's not what being solved here, neither this patch will compromise it. Running in a single thread makes the run way slower but gives you a save ride to prevent shared memory usage from multiple JSC calls.

You should also consider Test262 has only unit tests - as a maintainer I can guarantee that - and we don't run anything such as stress tests or anything relying on performance.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20180529/e3874b5f/attachment.html>