[webkit-gtk] Rationale for disabling pixel tests on bots

[Romain Bellessort]

Hi,

For several years (apparently since 2012), pixel tests have been disabled
when running tests on bots (see e.g. "Pixel tests disabled" in [1]). There
is an option to run them locally (-p), but I was wondering what was the
rationale for disabling them.

Based on what I found, the reason seems to be that running pixel tests on
bots has a high processing cost. In addition, in most cases, this cost is
not needed as considered features may be tested through reftests (hence
disabling pixel tests on bots is not a big issue as they can generally be
avoided).

Would you say this is correct, or are there other reasons?

Thanks,
Romain.

[1]
https://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release%20%28Tests%29/builds/2382/steps/layout-test/logs/stdio
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-gtk/attachments/20170803/2939ecb7/attachment.html>