[webkit-dev] generic test expectations?

[Glenn Adams]

On Tue, Nov 13, 2012 at 1:02 PM, Tony Chang <tony at chromium.org> wrote:

> On Tue, Nov 13, 2012 at 11:35 AM, Dirk Pranke <dpranke at chromium.org>wrote:
>
>> On Tue, Nov 13, 2012 at 11:29 AM, Glenn Adams <glenn at skynav.com> wrote:
>> >
>> > On Tue, Nov 13, 2012 at 12:09 PM, Dirk Pranke <dpranke at chromium.org>
>> wrote:
>> >>
>> >> We don't currently support port-specific reftests (or at least, not
>> >> very well), and we certainly should be trying to minimize where they
>> >> occur.
>> >
>> >
>> > Hmm, I actually used port specific reftest expectation files in a recent
>> > patch [1] (since rolled out), and it appeared to work (as a way to
>> > effectively rebaseline those expectations). So something seems to be
>> > working.
>> >
>> > [1] http://trac.webkit.org/changeset/133529
>> >
>>
>> I expect it'll sort of work, but it won't be robust; you may hit weird
>> behavior and/or bugs. We really haven't beaten on this aspect of
>> things, and I don't know yet how much we want to.
>
>
> I don't think we should support port specific ref test results.  That kind
> of misses the point of using a ref test in the first place.  I mean, you
> may as well check in port specific pixel results which are easier to review
> for correctness.
>
> It may be the case that a ref test is not appropriate for what you're
> trying to test.
>

In the case of line break behavior, using reftests seem better than pixel
tests, since there is less need for port-specific expectations. If I can
come up with a text based approach (perhaps using range boundary rects),
then I'll do so, but in the mean time, reftests seem a better option,
especially for defining correctness based expectations (instead of merely
regression based expectations). But we are straying from the original topic
of this thread.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20121113/e6adebd7/attachment.html>