[webkit-dev] Baselines ignoring metrics

[Dirk Pranke]

After some limited amount of thought, I'm inclined to agree with
Ryosuke and Ossy here. If we have tests that don't depend on checking
the metrics, can they just be dumpAsText tests or reftests instead?

I thought the initial motivation for --ignore-metrics was for new
ports to at least confirm that the render trees had the right "shape"
even if they didn't have the right metrics. It's unclear to me if this
has ever been useful for people, though.

On a related note (which I thought I mentioned in some
ignore-metrics-related bug, but apparently at least not this one),
I've been working on various render-tree-parsing-related features, and
I've been thinking that it might be nice to have a smarter diff that
could distinguish between fields that actually rendered differently
due to different intrinsic metrics vs. fields that rendered
differently because of different derived metrics (e.g., two text
fields are different sizes because the text renders differently vs.
two containing blocks that are also different because they contain the
different text fields; if both container and containing block are off
by one pixel, on the the contained block's diff is really
interesting). But, I don't think (but am not yet 100% sure) that the
existing rendertree format contains the information needed to do this
accurately.

-- Dirk


On Wed, Sep 19, 2012 at 1:08 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:
> As ossy has suggested in the bug, we should just convert those tests that
> don't require metrics equivalency to dumpAsText tests.
>
> On Sep 19, 2012 2:53 AM, "Allan Sandfeld Jensen" <kde at carewolf.com> wrote:
>>
>> Hello WebKit,
>>
>> Some time ago I wrote a patch that would make the test runners detect
>> baseline
>> that had been generated with the --ignore-metrics flag, see
>> https://bugs.webkit.org/show_bug.cgi?id=94815 . This makes it possible to
>> generate baselines that would always be testing ignoring metrics, and
>> which
>> would help consolidate more expected results between platforms - see the
>> second patch in the bug for an example of that.
>>
>> The cases I expect this to make sense in, are tests for css selectors,
>> parsing
>> and other places the final layout is not part of what is being tested.
>> Ideally
>> such new tests should be written using dumpText, but we have many old
>> test-
>> cases that has not been written that way, and many more that are imported
>> from
>> from external test suites which never will be.
>>
>> Should we apply the patch and start using ignore-metric baselines, and
>> should
>> we make some guidelines for what kind of tests this should be allowed on?
>>
>> Best regards
>> `Allan Sandfeld
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo/webkit-dev
>