[webkit-dev] Increasing the number of cross-platform/port expected results

[Maciej Stachowiak]

On Mar 1, 2010, at 5:49 PM, Ojan Vafai wrote:

> On Tue, Feb 23, 2010 at 3:26 PM, Eric Seidel <eric at webkit.org> wrote:
> On Tue, Feb 23, 2010 at 2:19 PM, Darin Adler <darin at apple.com> wrote:
> > But in practice pixel results are often ignored entirely. I think  
> that reftest-style tests if done right could be a great addition.
>
> Also, there are zillions of render-tree-dump tests which could easily
> be converted to plain-text tests.  No one has yet written a tool to
> visualize how many tests we have with platform-specific results, but I
> would expect many of those platform-specific results could be removed
> by converting more existing tests to dump as text.
>
> The vast majority of plain-text tests don't end up needing platform- 
> specific results. Also, they run faster.
>
> In the past there was some resistance to converting older tests to  
> dumpAsText out of fear of losing test coverage. Could we instead try  
> to make it so that the default is dumpAsText? That would go a long  
> way in making new tests be dumpAsText. The difficulty there, of  
> course, is actually doing it since there are so many tests that  
> would need to be modified (i.e. current render dump tests would need  
> something like a dumpRenderTree() call).
>
> Implementation difficulties aside, does that sound like a good idea?

On the one hand, it's good to shift the default. On the other hand,  
many render tree dumping tests do not currently need to run any  
JavaScript script, while many text-only tests do. I would almost  
suggest having a per-directory default but that might be too confusing  
and mysterious.

Question: are text-only tests really faster, in non-pixel mode? I  
would be surprised if there was a significant different, and it may  
point to something we can optimize.

Regards,
Maciej

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20100301/ae03e920/attachment.html>