[webkit-qt] QtWebKit trunk development after 5.0 release - questions

[Simon Hausmann]

On Monday, February 18, 2013 11:36:43 AM Osztrogonác Csaba wrote:
> Hi,
> 
> Simon Hausmann írta:
> >> build.webkit.org
> >> =================
> >> 
> >> - Qt Linux Release - 32 bit WK1 only build + layout tests + API tests
> > 
> > This doesn't have WK2 anymore because of the WK2 lockdown, right?
> 
> Exactly. Before WK2 lockdown, all Qt bots built WK2 too. But nowadays
> WK2 build is broken regularly (day by day) for 10-50 revisions. In this
> case it would be painful to lose the ability of catching layout test
> regressions at least with WK1 builds.

Okay, fully agreed.
 
> >> performance bots:
> >> ------------------
> >> - Qt Linux 64-bit Release (Perf) - WK1 only build and perf tests
> >> - Qt Linux 64-bit Release (WebKit2 Perf) - WK2 build and perf tests
> >> 
> >> The performance results are collected by perf-o-matic server -
> >> http://webkit-perf.appspot.com , but this tool is not user friendly
> >> to catch performance regressions, because you have to select tests
> >> one by one and check them manually. Nowadays they collect performance
> >> results, but in these circumstances we need someone who actively checks
> >> them. Do we want to appoint someone, or reuse these resources for more
> >> important tasks?
> > 
> > What's a good frequency for checking? How much work is it? Can it be done
> > once a week with an effort of say 30 minutes?
> 
> Unfortunately there isn't a good choice now. Unfortunately you have to
> check tests one by one on http://webkit-perf.appspot.com if you would
> like to catch a performance regression. I don't think if anybody's dream
> is this monotonic slave work :) In my opinion it would be better to
> improve the appspot and/or implement a new one which is less glittering,
> but more useful to catch performance regressions _automatically_.

Okay. In that case perhaps those bots should be removed to lighten the 
maintenance on your end?

I recall that we found a few regressions with this in the past, but I don't 
remember exactly how much effort was involved in finding them. Either way I 
fully agree that monotonic slave work is nobody's dream and I think that we 
must avoid it.
 
> >> offline bots:
> >> --------------
> >> - Qt Linux ARMv7 Release - It is offline, because we have a same
> >> 
> >>    bot on build.webkit.sed.hu to build ARM binaries for the tester bot.
> >> 
> >> These bots were not used for a long time. We suggest to
> >> delete them completely from build.webkit.org waterfall.
> >> What do you think about it?
> > 
> > Sounds good. Although... wouldn't it be nice to have the Qt Linux ARMv7
> > Release bot building on build.webkit.org, for better visibility?
> 
> We don't have two ARM bots to avoid duplication of the same
> bot. Unfortunately the ARM tester slave isn't in a good shape
> to move it to build.webkit.org and has some tricky local hack
> now which makes the migration impossible (for exmaple: the tester
> slave runs on x86 to reduce the runtime, but ssh to the board to call
> run-webkit-tests with some additional flag, etc.)
> 
> http://build.webkit.sed.hu/builders/ARMv7%20Linux%20Qt5%20Release%20%28Test%
> 29?numbuilds=200
> 
> And unfortunately the builder and tester slaves must be on the same
> buildbot master, because a builder on build.webkit.org can't trigger
> the tester slave on build.webkit.sed.hu and provide the binary WebKit
> for it.

Okay, I see. That makes perfect sense :). Then let's just have one set on 
build.webkit.sed.hu, if that works best for you guys.
 

Simon