[webkit-dev] Filtering results on wpt.fyi, Safari-specific failures

Robert Ma robertma at chromium.org
Fri Mar 29 10:16:05 PDT 2019

On Mon, Feb 25, 2019 at 8:49 AM Philip J├Ągenstedt <foolip at chromium.org>

> I'd like to point out right away that diagnosing reftest failures is
> currently cumbersome because we don't store the screenshots. This is
> also a work in progress:
> https://docs.google.com/document/d/1IhZa4mrjK1msUMhtamKwKJ_HhXD-nqh_4-BcPWM6soQ/edit?usp=sharing
> Until that has launched, I would recommend ignoring reftest failures
> if the cause of failure isn't obvious.

Great news! Reftest screenshots are now available on wpt.fyi. No more
guesswork for why a reftest fails!

For example, this
is one of the Safari-only reftest failures you can find using the search
link posted earlier. Now you can click the "compare" button (you might need
to force-reload the page to see it) to view the screenshots. This example
looks like a genuine failure, while some others are probably caused by font
antialiasing/kerning (they should most likely use the Ahem font instead).

We are also working on another feature to triage the failures
(e.g. to mark a test as a genuine failure and link it to bug trackers, or
as flaky/broken). Stay tuned!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20190329/7ea0b4f3/attachment.html>

More information about the webkit-dev mailing list