<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Nov 18, 2015, at 9:04 AM, Darin Adler <<a href="mailto:darin@apple.com" class="">darin@apple.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class=""><blockquote type="cite" class="">On Nov 18, 2015, at 4:36 AM, Carlos Alberto Lopez Perez <<a href="mailto:clopez@igalia.com" class="">clopez@igalia.com</a>> wrote:<br class=""><br class="">Some reference tests give a 1-pixel or very few pixel differences [1].<br class=""></blockquote><br class="">Why? We need to understand why.<br class=""><br class=""><blockquote type="cite" class="">Should we tolerate a few pixel differences for reftests ?<br class=""></blockquote><br class="">I don’t think we should. I think we should look more deeply into why these differences exist and consider writing the reference tests differently. We should not add tolerance when we don’t yet know why the results are different.<br class=""></div></div></blockquote></div><br class=""><div class="">There are some well-understood reasons why a test might not exactly match its reference. One is that the test uses compositing layers to do clipping, but the reference just clips with drawing, and these are not expected to give a pixel-perfect match.</div><div class=""><br class=""></div><div class="">I proposed a way to allow a test to specify a custom tolerance in <a href="https://bugs.webkit.org/show_bug.cgi?id=149828" class="">https://bugs.webkit.org/show_bug.cgi?id=149828</a>. If we had this, it would allow me to fix <a href="https://bugs.webkit.org/show_bug.cgi?id=142258" class="">142258</a> and <a href="https://bugs.webkit.org/show_bug.cgi?id=146523" class="">146523</a>.</div><div class=""><br class=""></div><div class="">Simon</div><div class=""><br class=""></div></body></html>