[webkit-dev] Iterating SunSpider

[Maciej Stachowiak]

On Jul 7, 2009, at 6:43 PM, Mike Belshe wrote:

>
> (There are other benchmarks that use summation, for example iBench,  
> though I am not sure these are examples of excellent benchmarks. Any  
> benchmark that consists of a single test also implicitly uses  
> summation. I'm not sure what other benchmarks do is as relevant of  
> the technical merits.)
>
> Hehe - I don't think anyone has iBench except apple :-)

This is now extremely tangential to the original point, but iBench is  
available to the general public here: <http://www.lionbridge.com/lionbridge/en-US/services/software-product-engineering/testing-veritest/benchmark-software.htm 
 >

> A lot of research has been put into benchmarking over the years;  
> there is good reason for these choices, and they aren't arbitrary.   
> I have not seen research indicating that summing of scores is  
> statistically useful, but there are plenty that have chosen  
> geometric means.


I think we're starting to repeat our positions at this point, without  
adding new information or really persuading each other.

If you have research that shows statistical benefits to geometric mean  
scoring, or other new information to add, I would welcome it.


Regards,
Maciej