[Webkit-unassigned] [Bug 168221] New: Nwtr unexpectedly passes mismatch ref test if the hashes doesn't match but no diff

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=168221

            Bug ID: 168221
           Summary: Nwtr unexpectedly passes mismatch ref test if the
                    hashes doesn't match but no diff
    Classification: Unclassified
           Product: WebKit
           Version: WebKit Nightly Build
          Hardware: Unspecified
                OS: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
          Assignee: webkit-unassigned at lists.webkit.org
          Reporter: Hironori.Fujii at sony.com
                CC: ap at webkit.org, lforschler at apple.com, rniwa at webkit.org
            Blocks: 168033

Nwtr unexpectedly passes mismatch ref test if the hashes doesn't match but no diff

There is a logic error in comparing mismatch ref test images.
In mismatch ref test images, it should be failed if two images has no diff.
But, if the hashes are different, nwtr unexpectedly pass the mismatch ref test.

It is difficult to test this bug.
First, you need to find a *match* ref test which does not match the hashes.
For example, in GTK port, I found a following test is the case:

> fast/css/sticky/sticky-left-percentage.html -> ref test hashes didn't match but diff passed

Next, convert this match ref test to a mismatch ref test:

> mv LayoutTests/fast/css/sticky/sticky-left-percentage-expected{,-mismatch}.html

And, run the test. In this time, it should fail:

> $ ./Tools/Scripts/run-webkit-tests --gtk --release --no-new-test-results fast/css/sticky/sticky-left-percentage.html -v

But, passed:

> [1/1] fast/css/sticky/sticky-left-percentage.html passed

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.webkit.org/pipermail/webkit-unassigned/attachments/20170213/a7c03726/attachment.html>