[webkit-dev] Pixel test differences between Leopard and Snow Leopard

[James Robinson]

Thanks for the feedback.  I've filed the following bugs for the different
steps involved (rebaselining, moving expectations, etc).

https://bugs.webkit.org/show_bug.cgi?id=39317
https://bugs.webkit.org/show_bug.cgi?id=39319
https://bugs.webkit.org/show_bug.cgi?id=39325

In the process of updating the Leopard baselines for tests that pass on Snow
Leopard I've already discovered (and filed) several repaint regressions on
Leopard involving animations and reflections.

On Tue, May 18, 2010 at 8:27 AM, Ojan Vafai <ojan at chromium.org> wrote:

> On Mon, May 17, 2010 at 3:59 PM, Simon Fraser <simon.fraser at apple.com>wrote:
>
>> On May 17, 2010, at 3:44 PM, James Robinson wrote:
>>
>> After this is done it should be possible to run the full layout test suite
>> with --tolerance=0 on Snow Leopard and have no failures.  Assuming this
>> happens, I would like to switch the default value of --tolerance from 0.1 to
>> 0.0 to try to keep regressions from creeping back in.
>>
>> You may find that some of the text antialiasing differences are
>> hardware-dependent, so I'm not sure if a tolerance of 0 would be possible.
>>
>
> I'd really like to see us default to a tolerance of 0 for at least one mac
> platform. Snow Leopard makes sense to be that platform. If we find that
> there are antialiasing differences that are hardware-dependent, then we
> should consider the hardware on the bots the default and, if
> possible, dynamically detect different hardware to set a higher tolerance.
>

Yes, I'm hoping that we'll be able to somehow limit this to a subset of
tests and/or hardware configurations if needed.  At a minimum we should be
able to make the tolerance much smaller than 0.1% (which is large enough to
paper over real regressions in some of our current tests).

- James

>  Does this sound like a good plan?  The biggest impact on the project will
>> be a number of large commits in LayoutTests/platform/ to do the svn move and
>> svn add operations to move results from platform/mac -> platform/mac-leopard
>> and to add new Snow Leopard results to platform/mac.  This can be done over
>> a weekend and mostly by a script to try to minimize impact.
>>
>> This sounds like a great short-term plan. I think a longer-term plan would
>> be to move as many tests to ref-tests as possible, which eliminates any
>> text-aliasing issues.
>>
>
> I fully support this plan.
>
> Ojan
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20100519/db761a5e/attachment.html>