[webkit-dev] Increasing the number of cross-platform/port expected results

[Ojan Vafai]

On Mon, Mar 1, 2010 at 6:40 PM, Maciej Stachowiak <mjs at apple.com> wrote:

> On Mar 1, 2010, at 5:49 PM, Ojan Vafai wrote:
>
> On Tue, Feb 23, 2010 at 3:26 PM, Eric Seidel <eric at webkit.org> wrote:
>
>> On Tue, Feb 23, 2010 at 2:19 PM, Darin Adler <darin at apple.com> wrote:
>> > But in practice pixel results are often ignored entirely. I think that
>> reftest-style tests if done right could be a great addition.
>>
>> Also, there are zillions of render-tree-dump tests which could easily
>> be converted to plain-text tests.  No one has yet written a tool to
>> visualize how many tests we have with platform-specific results, but I
>> would expect many of those platform-specific results could be removed
>> by converting more existing tests to dump as text.
>
>
> The vast majority of plain-text tests don't end up needing
> platform-specific results. Also, they run faster.
>
> In the past there was some resistance to converting older tests to
> dumpAsText out of fear of losing test coverage. Could we instead try to make
> it so that the default is dumpAsText? That would go a long way in making new
> tests be dumpAsText. The difficulty there, of course, is actually doing it
> since there are so many tests that would need to be modified (i.e. current
> render dump tests would need something like a dumpRenderTree() call).
>
> Implementation difficulties aside, does that sound like a good idea?
>
> On the one hand, it's good to shift the default. On the other hand, many
> render tree dumping tests do not currently need to run any JavaScript
> script, while many text-only tests do. I would almost suggest having a
> per-directory default but that might be too confusing and mysterious.
>

Yeah, I thought about having per-directory defaults too. There are some nice
things about that. For example, the editing tests mostly want something in
between dumpAsText and a render dump. It would be great if we could default
those to dumpAsMarkup (https://bugs.webkit.org/show_bug.cgi?id=26501).

But, I agree, it's probably too confusing.


> Question: are text-only tests really faster, in non-pixel mode? I would be
> surprised if there was a significant different, and it may point to
> something we can optimize.
>

I just meant that they are faster assuming you're running pixel tests.
Ideally we'd head in the direction of running pixel tests by default one
day. :)

Ojan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20100301/ec8cb1bc/attachment.html>