[webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

[Benjamin Poulain]

On Sat, Feb 9, 2013 at 1:24 AM, <noam.rosenthal at nokia.com> wrote:

>    I did not say they cannot be tested with the two methods suggested :)
> It's more  about a preference to move some of those decisions from the test
> infrastructure to the test itself, but potentially those bugs could be
> tested in either way.
> Examples for bugs I've encountered/reviewed and can use better in-motion
> testing (note that those are specific to Qt/EFL, but I'm sure there are
> tons of bugs like this that come up for Apple/Google as well)
> http://trac.webkit.org/changeset/140825
> http://trac.webkit.org/changeset/142112
> http://trac.webkit.org/changeset/134953
> https://bugs.webkit.org/show_bug.cgi?id=109179
>
>  Controlling the clock and programmatically sampling the end result would
> definitely make those more testable, but of course any progress in this
> area would be beneficial and my preference to a canvas-based API is more of
> an opinion.
>

To explain my concerns:
Sometime, I look at a failing test, and think: "what the f**k is this
supposed to test?". Then I have to dig a ton of logs, and finally read the
change to understand a the single JS statement in the whole script that
make the test useful.

This is the situation I am afraid with a solution where pixels would be
evaluated from JavaScript. You can test, but you lack visibility why
something is correct or incorrect. Text tests, ref-tests and pixel tests
have a great readability, you can quickly evaluate their correctness. This
is important in my opinion, I don't think we want more opaque output like
the RenderTree dump.

I am not against your plan if the readability of the tests can be good. I'd
be happy to review patches toward testing dynamic changes in webpages.

Benjamin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20130209/a18d20c8/attachment.html>