[webkit-dev] Testing feature suggestion: animation/interaction pixel-results "on the fly"

[Dirk Pranke]

On Wed, Feb 13, 2013 at 11:07 PM, Benjamin Poulain <benjamin at webkit.org> wrote:
> On Wed, Feb 13, 2013 at 9:38 PM, Dongsung Huang <luxtella at company100.net>
> wrote:
>>
>> I like this idea. I cannot find any harm if we have this functionality.
>
>
> Those changes are not harmless. There are people monitoring tests results
> full time in order to keep WebKit in good shape. No other part of WebKit
> require continuous attention.
>

I'm sorry, but either I don't understand Dongsung's suggestion, or I
don't understand your criticism. What does this sentence paragraph
mean? Are you suggesting someone needs to look at this type of test
full time in a way that we don't have to look at the other tests?


>>
>> Case 1: CSS Filters & Shaders
>> I wanted this test functionality when I commented
>> http://webkit.org/b/97859#c19
>> If I want to make gaussian blur test, I prefer using 'getPixel' test as
>> follows,
>
>
> Why wasn't a ref-test a better solution in this particular case?
>

I'm curious, what would you imagine the ref test contains?

>> Case 2: Fixed Position Element
>> [...]
>> function repeatedlyCalledDuringScrolling() {
>>     ASSERT(getPixel(15, 9) == white);
>>     ASSERT(getPixel(15, 10) == green);
>>     ASSERT(getPixel(9, 15) == white);
>>     ASSERT(getPixel(10, 15) == green);
>>     ....
>> }
>
>
> I think this shows what I said about correctness and readability:
> -Asserting the correctness of the test and the result becomes close to
> impossible for the reader. One has to review the full code to have a chance
> of understanding an error.
> -You cannot cover non trivial cases (images, text, form elements, etc).
> -And it is inefficient. You have to render each frame on the UIProcess, move
> it to the WebProcess, and box it for JavaScript to process (with pixel
> format conversions depending on your graphics system)
>
> Of the ideas raised, I think this is one of my least favorite for testing
> fixed positioning.
>

Isn't his suggestion the equivalent of what we do today in text-only
tests? i.e., printing "pass" or "fail" and making you have to look at
the test itself to see what's being tested?

If the correctness of the rendering depends on those 4 specific pixels
having those four specific values, how exactly are you going to verify
that by looking at it?

Again, I think I'm just not understanding you here?

-- Dirk