[webkit-qt] Does QtWebKit API tests need attention?

Ademar Reis ademar.reis at openbossa.org
Wed Jun 1 07:15:55 PDT 2011


2011/6/1 Osztrogonac Csaba <oszi at inf.u-szeged.hu>:
> Hi,
>
> Here is the master bug for fixing API tests:
> https://bugs.webkit.org/show_bug.cgi?id=38654
>
> I agree with Zoltan, always orange bot is absolutely useless
> for catching new regressions. (now 7 tests fail)
>
> I propose that we should mark all failing tests as expected fail
> and file bug reports block the master bug. (I did it for dozens
> API test before.)
>
> And when the API tests are green, first we should make one of our
> internal buildbot (http://build.webkit.sed.hu) stricter. I mean
> that only one failing API tests should make the build red.

Yep, I completely agree.

The problem with some API tests is that (because of historical lack of
care with them), several of them fail now, for inumerous reasons. Even
worse, some APIs are broken or ambiguous and fixing them will require
behavior change, which may be seen as a regression from previous
releases. Some fixes on corner cases or obscure APIs are also not
trivial and fixing them is not worth it right now (we better spend our
limited time fixing crashes or broken basic features, for example).

Anyway, in summary, part of the release criteria is to have all bots
green (in layout tests and API tests), so we'll either fix them or
mark as expected fail before the 2.2 release.

Cheers,
  - Ademar

-- 
Ademar de Souza Reis Jr. <ademar.reis at openbossa.org>
Nokia Institute of Technology


More information about the webkit-qt mailing list