[webkit-dev] IMAGE+TEXT WAS: TestExpectations syntax changes, last call (for a while, at least) ...

[Ojan Vafai]

On Thu, Jun 14, 2012 at 9:20 PM, Maciej Stachowiak <mjs at apple.com> wrote:

>
> On Jun 14, 2012, at 9:06 PM, Adam Barth <abarth at webkit.org> wrote:
>
> > On Thu, Jun 14, 2012 at 9:02 PM, Ojan Vafai <ojan at chromium.org> wrote:
> >>
> >> Seems like it will be a common error to mark a reftest failure as
> >> ImageOnlyFail and then be confused why it's not working, no?
> >
> > Maybe that can be solved with another name, like PixelOnlyFailure.
>

I'm OK with trying it this way. We can always have another
<strike>bikeshed</strike> fruitful discussion if it turns out to be a
frequent cause of confusion in practice.

Not sure one name is any more clear than the other. PixelOnlyFailure seems
fine to me since others have expressed a preference in the past for Pixel
over Image.


> That sounds good. We could also make it an error to apply PixelOnlyFailure
> (or what have you) to a text-only test, a reftest, or an audio test. Error
> in the sense that it would be reported as a failure, with an informative
> diagnostic saying it does not apply.
>

We already have a mechanism for "errors" like this. They are reported when
you run the tests or when you run with --lint-test-files. At least on the
chromium bots, this runs as a separate step that turns red when when you
cause a lint failure. That way errors get noticed and addressed quickly
(the lint step takes ~2 seconds to run).

Ojan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120614/89aa0c43/attachment.html>