[webkit-dev] adding webkit perf bots?

[Ojan Vafai]

Chromium has a bunch of perf bots that we run continuously. Most of these
are just page cyclers that loop through pages. They're grouped by different
types. You can see some graphs here:
http://build.chromium.org/buildbot/perf/dashboard/overview.html. The more
stable bots will also turn red on perf/memory regressions.

This is really useful data that it would be awesome to have at a more
granular level than each time chromium pulls in a new webkit revision. Also,
it would be great to have this upstream so that it doesn't depend on
chromium engineers to notice and help run the tests.

This came up recently due to a perf/memory regression from
http://trac.webkit.org/changeset/57215 on Mac loading international pages
(see
http://build.chromium.org/buildbot/perf/mac-release-10.5/intl2/report.html?history=150&rev=-1).
With that data we were able to find the root problem and have a path forward
for fixing it https://bugs.webkit.org/show_bug.cgi?id=37292.

What would it take to support these (the ones that test web page perf) in
upstream WebKit?

Some initial thoughts:
1. Machines for the new bots.
2. Make the page cyclers work with Safari (DRT?). I imagine this is not too
hard.
3. Make the data available. Currently, these are on Chrome's internal svn
repo. I think this is due to fear of copyright infringement if we were to
publicly distribute them (i.e. a copy of someone else's copyrighted
website). I don't know the details here and IANAL, so I don't know how hard
this is to workaround. Seems like we ought to be able to figure something
out though.

Any interest? Anyone willing to do the grunt work?

Maybe this is a good topic for the webkit conference.

Ojan


Abberviated #webkit discussion:
[11:29am] smfr: why doesn't webkit have data like that?
[11:30am] smfr: ojan: it sucks that such data are not available via
webkit.org
[11:30am] ojan: smfr: i agree
[11:30am] smfr: we should be measuring page load and memory use for every
build
[11:30am] ojan: smfr: no argument from me
[11:30am] dglazkov: smfr, ojan: we totally should.
[11:31am] Catfish_Man: mmm delicious delicious metrics
[11:31am] smfr: any volunteers?
[11:31am] dglazkov: I volunteer ojan
[11:32am] ojan: dglazkov: i unvolunteer myself!
[11:32am] dglazkov: then I volunteer smfr
[11:32am] ojan: but i fully support someone else moving these tests to
webkit
[11:32am] smfr: we need a good server-side hacker
[11:32am] smfr: that's not me
[11:33am] dglazkov: so would it be a matter of hooking up DRT to run various
tests and putting up a bot?
[11:33am] enrica: great project for an intern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20100409/dcc3f1da/attachment.html>