[webkit-dev] Making performance tests in WebKit more useful
rniwa at webkit.org
Fri Mar 8 02:29:22 PST 2013
On Fri, Mar 8, 2013 at 2:24 AM, Benjamin Poulain <benjamin at webkit.org>wrote:
> On Fri, Mar 8, 2013 at 2:15 AM, Ryosuke Niwa <rniwa at webkit.org> wrote:
>> Three major problems with existing WebKit performance tests are:
>> 1. There are too many tests to run
>> 2. Some tests have variances that are too high to be of any use
>> 3. Some tests are too specific to be of any general use
> If you find what differentiate the good tests from the bad tests, it would
> be good to create a wiki page explaining what to be careful about to make a
> good test.
Yeah. We need to spend more time analysing tests in categories 2 & 3.
One thing that have bitten my ass in the past is the JIT being smart and
> removing some "dead" test code because it has no side effect. I am sure
> there are plenty of mistake like this to be avoided.
I was always curious about this. I wonder if there’s a good way to amortize
the effect so that we can measure relialistic number. The last thing I want
is to introduce a lot of measurement biases to lower the variance and end
up measuring something unrealistic.
- R. Niwa
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the webkit-dev