[webkit-dev] Handling failing reftests
dpranke at chromium.org
Thu Apr 12 10:57:06 PDT 2012
On Thu, Apr 12, 2012 at 12:36 AM, Ryosuke Niwa <rniwa at webkit.org> wrote:
> On Thu, Apr 12, 2012 at 12:12 AM, Osztrogonac Csaba <oszi at inf.u-szeged.hu>
>> I ran into a failing reftest and I didn't find a proper way for handling
>> fast/multicol/cell-shrinkback.html is a new reftest introduced in
>> http://trac.webkit.org/changeset/113738 .
>> But unfortunately it fails almost everywhere (Chromium, GTK, Qt).
> What is the failure? Is there a bug?
>> Chromium guys added platform specific png files into
>> platform/chromium-linux and platform/chromium-win
>> directories - http://trac.webkit.org/changeset/113790
> That won't work.
>> I tried to do same thing for Qt port, but the test still fails -
>> It seems NRWT ignores expected png and txt files for reftests at all.
> Right because it's a ref test.
>> And now I found that it is added to the test_expectations.txt for
>> So what is the proper way for handling failing reftests? In this case
>> isn't possible
>> if the reftest is incorrect, because it fails almost on all platform, but
> You can either skip the test or add a platform-specific reference file (e.g.
This is an interesting situation; does it make sense to require our
reftests to be generic, or is that unrealistic? I also wonder if we
should allow platform-specific pixel tests to coexist with
platform-specific reference tests (since you could obviously write a
platform-specific ref test that just displayed a single IMG, but the
tools don't make that particularly easy).
More information about the webkit-dev