[webkit-dev] ImageDiff tool for gtk platform

[Mark Rowe]

On 23/05/2008, at 00:39, Kalle Vahlman wrote:

> 2008/5/23 Mark Rowe <mrowe at apple.com>:
>>
>> On 22/05/2008, at 07:24, S Venkata Udai Kiran wrote:
>>
>>> Hi all,
>>> I was trying to understand how the Layout tests are being done in
>>> Webkit. I  went through the perl code 'run-webkit-tests' and found  
>>> that
>>> the script is using a tool name ImageDiff
>>> (WebKit/WebKitBuild/Release/Programs/ImageDiff) .But I could not  
>>> 'find'
>>> that tool in WebKit folder. As I understand that ImageDiff tool is
>>> necessary to do a pixel level diff for test images generated.I am
>>> looking at running individual tests using a small shell script  
>>> rather
>>> running all the tests at once. this will help me to know the  
>>> underlying
>>> things.  Please help.
>>
>> ImageDiff is currently only implemented for Mac and Windows.  It's  
>> not
>> needed to run the standard regression tests.  Pixel test results  
>> will differ
>> between platforms and software versions, so there is a lot of  
>> manual work
>> involved in verifying results when bringing up pixel results on a new
>> platform.  Focusing on text-only and render tree tests would  
>> provide much
>> better bang for your buck than pixel test results.
>
> That's very true. However, cairo aims at constant quality across
> output targets and has a similar utility in it's testing suite to
> verify that:
>
> http://gitweb.freedesktop.org/?p=cairo;a=tree;h=4bcc292bc335525f804db83f002692fb42a7b7c7;hb=8efb103600e9c034f8652f495d390d6a4178061e;f=test/pdiff
>
> So assuming that the layout code produces identical output on all
> platforms, testing cairo -backed platforms (gtk, win32...) for
> pixel-perfectness might make sense.

That Cairo utility doesn't test for pixel-perfectness, it tries to  
test that there are no perceptible differences.  It's an interesting  
idea and perhaps one that we should explore as an avenue to improve  
ImageDiff.  I don't think it would completely eliminate platform  
differences, though it should help (eg,  different available fonts  
could lead to different layout, different versions of the libraries  
used for complex text layout may calculate slightly different metrics,  
etc.).  There would also be a lot of manual work involved in verifying  
that the initial pixel results are correct (or at least identifying  
those that are known to be incorrect). The text-only results should be  
identical across platforms which makes them a lot easier to bring up  
on a new port.

- Mark

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2413 bytes
Desc: not available
Url : http://lists.webkit.org/pipermail/webkit-dev/attachments/20080523/99c40ba9/attachment.p7s