[webkit-dev] BigO correlation tests

[Nikos Andronikos]

> On 13 Apr 2016, at 4:22 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:
>
> I don’t think order of magnitudes tests ever quite worked JSC (it was added by Google so it obviously worked on V8).  If adjusting the threshold was all that was needed to make the tests pass, I see no reason how to change it.

Adjusting the threshold is one part, but we’re also improving the method by examining the slope to give a better indication of whether the test is running in constant time.

>
> However, what you proposed will only reduce the likelihood of type I errors (false positives).  You should also examine how it affects the likelihood of type II errors (false negative).  You might want to make some API artificially O(n^2) and make sure the test starts failing, etc…

That’s a fair comment (regarding false negatives), we can do that. But to be honest, I don’t have a lot of faith in the reliability of these tests anyway. Since they were added by Google, do you have any thoughts on whether we could remove them?

> You can probably look at the svn/git blame the change which added relevant tests in LayoutTests/perf/ and apply a reverse change locally to test that.

Sorry, I’m not following what you’re saying here.


Nikos
The information contained in this email message and any attachments may be confidential and may also be the subject to legal professional privilege. If you are not the intended recipient, any use, interference with, disclosure or copying of this material is unauthorised and prohibited. If you have received this email in error, please immediately advise the sender by return email and delete the information from your system.