[webkit-dev] Handling failing reftests

[Ryosuke Niwa]

My suggestion is to try to make it platform-independent but it's
significantly harder than rebaselining the results for pixel tests. As you
point out, it defeats one of the benefits of reftests if we end up
introducing platform-specific reference files.

- Ryosuke

On Thu, Apr 12, 2012 at 1:46 PM, Jacob Goldstein <jacobg at adobe.com> wrote:

> Right, but when those differences come to light, should the goal be to
> adjust the ref test to make it platform-independent, or is having
> platform-specific ref tests acceptable?  Doesn't that put us in the same
> situation as having platform-specific pixel tests?
>
> From: Ryosuke Niwa <rniwa at webkit.org>
> Date: Thu, 12 Apr 2012 13:43:26 -0700
> To: Jacob Goldstein <jacobg at adobe.com>
> Cc: Ojan Vafai <ojan at chromium.org>, Dirk Pranke <dpranke at chromium.org>,
> WebKit Development <webkit-dev at lists.webkit.org>
>
> Subject: Re: [webkit-dev] Handling failing reftests
>
> Right but you wouldn't know platform-specific issues until you land them.
> e.g. rounding errors, subtle font differences in edge cases, etc...
>
> On Thu, Apr 12, 2012 at 1:30 PM, Jacob Goldstein <jacobg at adobe.com> wrote:
>
>> Isn't the goal of writing a ref test that it is not platform specific?
>>
>> From: Ryosuke Niwa <rniwa at webkit.org>
>> Date: Thu, 12 Apr 2012 12:29:42 -0700
>> To: Ojan Vafai <ojan at chromium.org>
>> Cc: Dirk Pranke <dpranke at chromium.org>, WebKit Development <
>> webkit-dev at lists.webkit.org>
>> Subject: Re: [webkit-dev] Handling failing reftests
>>
>> On Thu, Apr 12, 2012 at 11:53 AM, Ojan Vafai <ojan at chromium.org> wrote:
>>>
>>> I agree that it's hard sometimes to construct reftests that work, but
>>> once you've done so, the cost on the project of maintaining the test is
>>> usually considerably lower than pixel tests.
>>>
>>
>> Not so sure. There are cases where we have these platform specific
>> failures for ref tests, and they're much harder to fix than rebaselining
>> pixel results.
>>
>> - Ryosuke
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120412/310909d0/attachment.html>