<html>
    <head>
      <base href="https://bugs.webkit.org/" />
    </head>
    <body><table border="1" cellspacing="0" cellpadding="8">
        <tr>
          <th>Bug ID</th>
          <td><a class="bz_bug_link 
          bz_status_NEW "
   title="NEW - Nwtr unexpectedly passes mismatch ref test if the hashes doesn't match but no diff"
   href="https://bugs.webkit.org/show_bug.cgi?id=168221">168221</a>
          </td>
        </tr>

        <tr>
          <th>Summary</th>
          <td>Nwtr unexpectedly passes mismatch ref test if the hashes doesn't match but no diff
          </td>
        </tr>

        <tr>
          <th>Classification</th>
          <td>Unclassified
          </td>
        </tr>

        <tr>
          <th>Product</th>
          <td>WebKit
          </td>
        </tr>

        <tr>
          <th>Version</th>
          <td>WebKit Nightly Build
          </td>
        </tr>

        <tr>
          <th>Hardware</th>
          <td>Unspecified
          </td>
        </tr>

        <tr>
          <th>OS</th>
          <td>Unspecified
          </td>
        </tr>

        <tr>
          <th>Status</th>
          <td>NEW
          </td>
        </tr>

        <tr>
          <th>Severity</th>
          <td>Normal
          </td>
        </tr>

        <tr>
          <th>Priority</th>
          <td>P2
          </td>
        </tr>

        <tr>
          <th>Component</th>
          <td>Tools / Tests
          </td>
        </tr>

        <tr>
          <th>Assignee</th>
          <td>webkit-unassigned&#64;lists.webkit.org
          </td>
        </tr>

        <tr>
          <th>Reporter</th>
          <td>Hironori.Fujii&#64;sony.com
          </td>
        </tr>

        <tr>
          <th>CC</th>
          <td>ap&#64;webkit.org, lforschler&#64;apple.com, rniwa&#64;webkit.org
          </td>
        </tr>

        <tr>
          <th>Blocks</th>
          <td>168033
          </td>
        </tr></table>
      <p>
        <div>
        <pre>Nwtr unexpectedly passes mismatch ref test if the hashes doesn't match but no diff

There is a logic error in comparing mismatch ref test images.
In mismatch ref test images, it should be failed if two images has no diff.
But, if the hashes are different, nwtr unexpectedly pass the mismatch ref test.

It is difficult to test this bug.
First, you need to find a *match* ref test which does not match the hashes.
For example, in GTK port, I found a following test is the case:

<span class="quote">&gt; fast/css/sticky/sticky-left-percentage.html -&gt; ref test hashes didn't match but diff passed</span >

Next, convert this match ref test to a mismatch ref test:

<span class="quote">&gt; mv LayoutTests/fast/css/sticky/sticky-left-percentage-expected{,-mismatch}.html</span >

And, run the test. In this time, it should fail:

<span class="quote">&gt; $ ./Tools/Scripts/run-webkit-tests --gtk --release --no-new-test-results fast/css/sticky/sticky-left-percentage.html -v</span >

But, passed:

<span class="quote">&gt; [1/1] fast/css/sticky/sticky-left-percentage.html passed</span ></pre>
        </div>
      </p>
      <hr>
      <span>You are receiving this mail because:</span>
      
      <ul>
          <li>You are the assignee for the bug.</li>
      </ul>
    </body>
</html>