[webkit-dev] Handling failing reftests

[Ryosuke Niwa]

On Thu, Apr 12, 2012 at 12:12 AM, Osztrogonac Csaba <oszi at inf.u-szeged.hu>wrote:
>
> I ran into a failing reftest and I didn't find a proper way for handling
> it.
> fast/multicol/cell-shrinkback.**html is a new reftest introduced in
> http://trac.webkit.org/**changeset/113738<http://trac.webkit.org/changeset/113738>.
> But unfortunately it fails almost everywhere (Chromium, GTK, Qt).
>

What is the failure? Is there a bug?

Chromium guys added platform specific png files into
> platform/chromium-linux and platform/chromium-win
> directories - http://trac.webkit.org/**changeset/113790<http://trac.webkit.org/changeset/113790>


That won't work.

I tried to do same thing for Qt port, but the test still fails -
> http://trac.webkit.org/**changeset/113869<http://trac.webkit.org/changeset/113869>
> It seems NRWT ignores expected png and txt files for reftests at all.
>

Right because it's a ref test.

And now I found that it is added to the test_expectations.txt for Chromium:
> http://trac.webkit.org/**changeset/113797/trunk/**
> LayoutTests/platform/chromium/**test_expectations.txt<http://trac.webkit.org/changeset/113797/trunk/LayoutTests/platform/chromium/test_expectations.txt>
>
> So what is the proper way for handling failing reftests? In this case
> isn't possible
> if the reftest is incorrect, because it fails almost on all platform, but
> Mac?
>

You can either skip the test or add a platform-specific reference file
(e.g. platform/qt/fast/multicol/cell-shrinkback-expected.html).

- Ryosuke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120412/d8718a45/attachment.html>