[webkit-dev] PSA: FAIL test expectation does not encompass MISSING, CRASH, or TIMEOUT

[Ryosuke Niwa]

Hi,

As much as it doesn't make any sense to me, FAIL test expectation doesn't
cover MISSING, CRASH, and TIMEOUT test expectations as it is currently
interpreted by new-run-webkit-tests. Meaning that

BUGRNIWA : some-pixel-test-i-am-adding.html = FAIL

is logically equivalent to

BUGRNIWA : some-pixel-test-i-am-adding.html = IMAGE TEXT IMAGE+TEXT

Thus, if you're adding a pixel test and only adding the corresponding
expected result to, say, Mac, then you need to add MISSING results to some
platforms (e.g. Qt, GTK+, etc...) that cannot find this expected result as
in:

BUGRNIWA : some-pixel-test-i-am-adding.html = MISSING

If you're expecting the test to fail with image, text, or image and text
failures, or pass, then do:

BUGRNIWA : some-pixel-test-i-am-adding.html = FAIL PASS

If you're expecting the test to timeout, crash, or, fail (image, text, or
image and text), or pass, then do:

BUGRNIWA : some-pixel-test-i-am-adding.html = TIMEOUT CRASH FAIL PASS

Best,
Ryosuke Niwa
Software Engineer
Google Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120606/3b4f85cb/attachment.html>