On Jul 26, 2007, at 11:13 AM, Andre-John Mas wrote:
What should the requirements for the tool be, other than being freely available?
I'm not sure these are requirements, but here are some of the things I'd like to see: - representative of real world performance - one way to make it a real-world test is to use content from real-world websites, but that's the rub because the copyright owner may not give us license to use it as we wish - easy to run in an automated fashion so we can post results on the buildbots - easy to run with performance analysis tools - possible to run in non-WebKit browsers too - no "axe to grind" -- this wouldn't be a test specifically designed to highlight a particular performance problem - doesn't change a lot over time -- it's hard to judge changes in performance of the engine if the test is changing too There are other more subtle requirements. If we have too many different performance tests, they are no good because we can't run them all. If benchmarks are created by unaffiliated third parties, they are even more valuable because people won't assume they are "slanted". -- Darin