[webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

[noam.rosenthal at nokia.com]

From: ext Benjamin Poulain <benjamin at webkit.org<mailto:benjamin at webkit.org>>
Date: Saturday, February 9, 2013 12:52 AM
To: Noam Rosenthal <noam.rosenthal at nokia.com<mailto:noam.rosenthal at nokia.com>>
Cc: "rniwa at webkit.org<mailto:rniwa at webkit.org>" <rniwa at webkit.org<mailto:rniwa at webkit.org>>, "webkit-dev at lists.webkit.org<mailto:webkit-dev at lists.webkit.org>" <webkit-dev at lists.webkit.org<mailto:webkit-dev at lists.webkit.org>>
Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

On Fri, Feb 8, 2013 at 3:16 PM, <noam.rosenthal at nokia.com<mailto:noam.rosenthal at nokia.com>> wrote:
The problem with dynamic features of the web like animations/interactions is that they're non-deterministic, or at least a lot less deterministic than static features of the web like layouts.
Ref tests, pixel tests etc. are tools built for deterministic testing: load a file, take a snapshot, compare against a result. Testing an animation (or a filter) needs to feel a lot more dynamic and expressive: Animate green boxes, make sure that they're within a particular range at particular points in time".

The tests also have to be deterministic and comprehensive. I am afraid of loosing both with the Render-to-Canvas approach.

Can you give concrete examples of the kind of bugs you are hunting, and why testing cannot use the two methods suggested?
I did not say they cannot be tested with the two methods suggested :) It's more  about a preference to move some of those decisions from the test infrastructure to the test itself, but potentially those bugs could be tested in either way.
Examples for bugs I've encountered/reviewed and can use better in-motion testing (note that those are specific to Qt/EFL, but I'm sure there are tons of bugs like this that come up for Apple/Google as well)
http://trac.webkit.org/changeset/140825
http://trac.webkit.org/changeset/142112
http://trac.webkit.org/changeset/134953
https://bugs.webkit.org/show_bug.cgi?id=109179

Controlling the clock and programmatically sampling the end result would definitely make those more testable, but of course any progress in this area would be beneficial and my preference to a canvas-based API is more of an opinion.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20130209/c8e8e1ee/attachment.html>