[Webkit-unassigned] [Bug 43255] New: Millisecond timing is too imprecise in SunSpider

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=43255

           Summary: Millisecond timing is too imprecise in SunSpider
           Product: WebKit
           Version: 528+ (Nightly build)
          Platform: PC
        OS/Version: All
            Status: UNCONFIRMED
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
        AssignedTo: webkit-unassigned at lists.webkit.org
        ReportedBy: pbiggar at mozilla.com


The largest part of the variability I mentioned in bug 43253 is that millisecond timing is insufficient for such short tests. I reduced the total variance in the tests by about 50% by using sub-millisecond timing. A few possible resolutions:

- Using submillisecond timing. This would mean checking for the presence of a Date.timeInMicroSeconds() function, or similar, which would have to be created by the vendors. This is simple to add to Firefox, I don't know about others.

- Increase the test lengths until they are no longer affected by this bug. The worst affected test is 
bitops-3bit-bits-in-byte, which typically runs in less than 1ms on my machine. bitops-bitwise-and, controlflow-recursive, crypto-sha1, math-spectral-norm, string-base64, and access-nbody all run in less than 10ms. The error due to this bug is highest in these tests.


I think the second resolution is bad, because the test runs would have to be increased to perhaps 100ms each to really solve the problem, and that makes the test runs very long. It also means that if engines get much better, we run into the problem again. By contrast, checking for a Date.timeInMicroSeconds() function is non-invasive, allows vendors to opt-in, and doesn't change the experience for anyone.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.