[webkit-qt] QtWebKit trunk development after 5.0 release - questions [Caution: Message contains Suspicious URL content]

[Balazs Kelemen]

Comments inlined.

On 02/12/2013 05:05 PM, Simon Hausmann wrote:
> On Monday, February 11, 2013 05:32:20 PM Árvai, Zoltán wrote:
>> Hi QtWebKit hackers,
>>
>> we started upgrading QtWebKit buildbots and EWS bots
>> today to Ubuntu 12.04 LTS + GStreamer 1.0 + Qt 5.0.1 .
> Lovely!
>   
>> Using GStreamer 1.0 will be mandatory soon, see this mail for details:
>> https://lists.webkit.org/pipermail/webkit-dev/2013-January/023246.html
>> (You can easily install GStreamer 1.0 on Ubuntu 12.04 - see
>>    https://trac.webkit.org/wiki/QtWebKitGardening for details)
>>
>> It is a high time to review which bots we still need, which bots we
>> don't need anymore. Or do we need new bots with different configuration?
>>
>> Currently we have the following bots:
>>
>> build.webkit.org
>> =================
>>
>> - Qt Linux Release - 32 bit WK1 only build + layout tests + API tests
> This doesn't have WK2 anymore because of the WK2 lockdown, right?

Lockdown or not, I think it's odd that the most visible standard bot is 
still testing WebKit1 while the development if focusing on WebKit2.

>
>> - Qt Linux Release minimal - 32 bit WK1 only build
>>     (good for catching broken ifdef guards)
>> - Qt Windows 32-bit Release - WK1/WK2 build
>>     (MS Win Server 2008 R2, MS Visual Studio 2010 Express)
>>
>> performance bots:
>> ------------------
>> - Qt Linux 64-bit Release (Perf) - WK1 only build and perf tests
>> - Qt Linux 64-bit Release (WebKit2 Perf) - WK2 build and perf tests
>>
>> The performance results are collected by perf-o-matic server -
>> http://webkit-perf.appspot.com , but this tool is not user friendly
>> to catch performance regressions, because you have to select tests
>> one by one and check them manually. Nowadays they collect performance
>> results, but in these circumstances we need someone who actively checks
>> them. Do we want to appoint someone, or reuse these resources for more
>> important tasks?
> What's a good frequency for checking? How much work is it? Can it be done once
> a week with an effort of say 30 minutes?

There must be a way to automate this, unless the performance tests don't 
have much value. Probably we should rise this on webkit-dev.

>   
>> offline bots:
>> --------------
>> - Qt Linux ARMv7 Release - It is offline, because we have a same
>>     bot on build.webkit.sed.hu to build ARM binaries for the tester bot.
>>
>> - Qt Windows 32-bit Debug - We used it when cross compiling on Linux was
>>     possible with MinGW (with Qt 4.8). But now the Windows build works
>>     with MSVC on native Windows and we don't have more Windows hardware.
>>
>> These bots were not used for a long time. We suggest to
>> delete them completely from build.webkit.org waterfall.
>> What do you think about it?
> Sounds good. Although... wouldn't it be nice to have the Qt Linux ARMv7
> Release bot building on build.webkit.org, for better visibility?

I agree. I don't think that those bots that are not visible to 
webkit.org have too much value.


>   
>> bots maintained by the community:
>> ----------------------------------
>> - Qt Linux MIPS32R2 LE Release
>> - Qt Linux SH4 Release
>> - Qt Mountain Lion Release - maintained by INdT
>>     (Is INdT still intereseted in QtWebKit on Mac?)
>>
>>
>> build.webkit.sed.hu
>> ====================
>> - x86-64 Linux Qt Release - 64 bit WK1 only build + layout/API tests
>> - x86-64 Linux Qt Debug - 64 bit WK1 only build + layout/API tests
>> - x86-64 Linux Qt Release WebKit2 (Amazon EC2) -
>>         64 bit build + WK2 layout/API tests
>> - x86-64 Linux Qt Release WebKit2 (Pixel Tests)
>>         64 bit build + WK2 UI process side pixel tests
>> - x86-32 Linux Qt Release WebKit2
>>         32 bit build + WK2 layout/API tests
>> - x86-64 Mountain Lion Qt Release(WebGL Tester)(maintained by zalbisser)
>>         64 bit build + fast/canvas/webgl WK2 tests + API tests
>> - x86-32 Linux Qt Release NRWT
>>         32 bit WK1 only builder + layout tests with 4 parallel threads
>>     (It is an experimental bot, because parallel testing is very flakey,
>>      on Qt, see https://bugs.webkit.org/show_bug.cgi?id=77730 and
>>      https://bugs.webkit.org/show_bug.cgi?id=106218 for details.)
>> - ARMv7 Linux Qt5 Release (Build) - WK1 only build for ARM traditional
>>     platform (ARM instruction set, not Thumb2)
>> - ARMv7 Linux Qt5 Release (Test) - JSC+layout tests on a Panda board
>> - x86-32 Linux Qt Debug - 32 bit WK1 only build + layout/API tests

I think this is a bit redundant set. I don't think that in a project 
like WebKit (which is pretty high level stuff with the exception of the 
JIT) 32 vs. 64 bit is so much a difference that we need to have a bot 
for every combination. For that reason I think one release and one debug 
tester is enough for a product (WK1 or WK2) if one of them is 32 bit and 
the other is 64. What would have more value in my point of view is to 
have a debug bot on webkit.org so people can see if there is an 
assertion fail on Qt after his change (which is very commonly a general 
problem that somehow not triggered on other ports). So, I would make the 
following changes (just to get up with a concrete plan that we can discuss):

- kill build.webkit.sed.hu - x86-64 Linux Qt Release - 64 bit WK1 only 
build + layout/API tests
$ move x86-64 Linux Qt Debug - 64 bit WK1 only build + layout/API tests 
to webkit.org
$ move build.webkit.sed.hu - x86-64 Linux Qt Release WebKit2 (Amazon 
EC2) - 64 bit build + WK2 layout/API tests to webkit.org
$ turn build.webkit.sed.hu - x86-32 Linux Qt Release WebKit2 32 bit 
build + WK2 layout/API tests to a debug bot and move to webkit.org
     the two above could also change word size, since 32 bit debug is tricky
- kill x86-32 Linux Qt Debug - 32 bit WK1 only build + layout/API tests

Br,
-kbalazs