[webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

[noam.rosenthal at nokia.com]

From: ext Ryosuke Niwa <rniwa at webkit.org<mailto:rniwa at webkit.org>>
I'd like to propose a solution, and would welcome some feedback on whether it's a good one...
The idea is that you would be able to programatically retrieve the current snapshot into a canvas ImageData, and then compare the pixel results with JavaScript in the LayoutTest. Something like:

I had similar thoughts but my counter "proposal" is to let DRT/WTR generate multiple actual results either in the form of multiple layers in PNG or multiple PNG images. The advantage of this latter approach is that we can have multiple reference files as well. e.g. if a green box is to be moved from (0,0) to (50,0) and to (100, 0), we could create three reference files that correspond to each state.

But perhaps there is a good reason you didn't choose this approach. Could you elaborate on the reason you picked this particular API?
The problem with dynamic features of the web like animations/interactions is that they're non-deterministic, or at least a lot less deterministic than static features of the web like layouts.
Ref tests, pixel tests etc. are tools built for deterministic testing: load a file, take a snapshot, compare against a result. Testing an animation (or a filter) needs to feel a lot more dynamic and expressive: Animate green boxes, make sure that they're within a particular range at particular points in time".
I think that when testing some of the dynamic features it's best to create a minimal mechanism such as retrieving snapshot pixels, giving more power to the tests themselves in a dynamic language like JavaScript, rather than try to force the tests to fit into a predefined set of features supported by the DRT/WTR infrastructure. Also this would allow some differences between the ports, which implement animations a bit differently, without constant rebaselines of reference results – e.g. A heuristic animation timing test may stay valid across platform even if the actual animation implementation is different.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20130208/112f3ab9/attachment.html>