[webkit-qt] About the fonts used for QtWebKit tests

[Osztrogonac Csaba]

Hi,

Caio Marcelo de Oliveira Filho írta:
> Disclaimer: I'm not a font expert, I just want to make an editing
> layout test to pass in Qt port, and it happens to depend on monospace
> fonts. :]

Me neither. :)

> Last week I've hacked a bit our font infrastructure in testing. I've
> found out that the current state is not good: we have two odd
> configuration files with a lot of indirection and unused information,
> and we also have with problems like not getting monospace font when
> tests asks for those fonts. This naturally cause some tests that
> depend the fixed size of the characters to fail, for example
> https://bugs.webkit.org/show_bug.cgi?id=50144.
> 
> After talking to elproxy and bbandix, I made a new configuration file
> (and corresponding changes for QtWebKit), tested by me in Qt 4.8, 5.0
> and 5.0-wk2 and also some tests by bbandix, that seems to do the right
> thing. The patches are here
> https://bugs.webkit.org/show_bug.cgi?id=85203 for you guys to take a
> look and review.

And will Qt 4.8, Qt 5.0 WK1 and Qt 5.0 WK2 testresults match after this change?
Now our problem is that Qt 4.8 and Qt 5.0 WK2 results match, but Qt 5.0 WK1
is different. Or it would be different without elproxy's testfonts hack.

> The problem is: this change means we need to do a big rebaseline of
> ~5000 tests (also listed in the previous patch). Not only monospace
> fonts changed but also some fonts that previously were wrongly
> assigned to our sans-serif, are now properly assigned to our serif
> font. Note that even though we have some pixel tests, not all of them
> matches the actual results even without the patch.
> 
> Brings me to question
> 
> 1) How we handle such a big rebaseline? How you guys did before?
> 
> - One person do it in a big rebaseline (I'm not very enthusiastic with
> this approach, although the rebasilne server tool seems very good).
> - Skip all those tests (in a separated section in the Skipped file)
> and "crowdsource" the work of rebaselining.
> - Another way?

Rebasing 5000 tests in one shot is absolutely impossible. I made 500
sized rebases previously, but I had zillion problem with it. It takes
long time to commit a big patch (~10-15 minutes) and it always fails
if someone else commit a change in LayoutTests dir during my commit,
because of conflicting LayoutTests/ChangeLog file. :-/

So landing all new baselines would take days. (And otherwise we should
check them one by one to avoid accidentally commit wrong expected files
and hide new bugs.) In this case skipping tests for 1-2-3 days are
absolutely reasonable.

> And since we are doing this rebaseline, another question...
> 
> 2) Given that we have a huge rebaseline ahead, what about changing our
> Nimbus family of fonts to the Liberation family of fonts
> (http://en.wikipedia.org/wiki/Liberation_fonts). I see two benefits:
> 
> - They share the metrics with the Arial/TimesNewRoman/CourierNew
> fonts, so we could _potentially_ share some expected results with
> other ports.

I like this idea. ;) (I don't know why we usethe Nimbus
family, but I'm sure there is a rationale for it.)

> - And even if we don't, the results at least will look similar, making
> our and other port developer life easier when assessing a patch and
> doing a rebaseline.

Just a question. What kind of (test)fonts use the other ports? Apple? GTK? Chromium?
Will our results be closer to them with this change? And what about Qt-Mac port?

> Note that these fonts are not to be distributed with Qt, just used by
> developers and robots. We can keep our version of them in
> testfonts.git.
> 
> 
> Long story short:
> (1) How we do this rebaseline?
> (2) Let's use the opportunity to migrate to Liberation family?


br,
Ossy