[webkit-reviews] review denied: [Bug 59078] Reftests should be skipped if pixel tests are disabled. : [Attachment 90690] skip-reftests-if-pixel-tests-are-disabled

[bugzilla-daemon at webkit.org]

Ojan Vafai <ojan at chromium.org> has denied Hayato Ito <hayato at chromium.org>'s
request for review:
Bug 59078: Reftests should be skipped if pixel tests are disabled.
https://bugs.webkit.org/show_bug.cgi?id=59078

Attachment 90690: skip-reftests-if-pixel-tests-are-disabled
https://bugs.webkit.org/attachment.cgi?id=90690&action=review

------- Additional Comments from Ojan Vafai <ojan at chromium.org>
View in context: https://bugs.webkit.org/attachment.cgi?id=90690&action=review

> Tools/Scripts/webkitpy/layout_tests/layout_package/test_runner.py:254
> +	       reftests_files = set(test_file for test_file in self._test_files
if self._port.is_reftest(test_file))
> +	       self._printer.print_expected('%d reftests will be skipped
because pixel tests are disabled.' % len(reftests_files))
> +	       self._test_files -= reftests_files

This is a lot of file-system reading just to determine a small number of tests
to read. The problem is that we're blocking running any tests on this. I'd
rather we not spit out the number of tests skipped and instead skip the test
when we're about to run it.