[webkit-dev] Increasing the number of cross-platform/port expected results

[Dirk Pranke]

On Tue, Apr 13, 2010 at 11:54 AM, Ojan Vafai <ojan at chromium.org> wrote:
> On Mon, Mar 1, 2010 at 7:53 PM, Maciej Stachowiak <mjs at apple.com> wrote:
>>
>> On Mar 1, 2010, at 6:46 PM, Ojan Vafai wrote:
>>
>> On Mon, Mar 1, 2010 at 6:40 PM, Maciej Stachowiak <mjs at apple.com> wrote:
>>>
>>> On the one hand, it's good to shift the default. On the other hand, many
>>> render tree dumping tests do not currently need to run any JavaScript
>>> script, while many text-only tests do. I would almost suggest having a
>>> per-directory default but that might be too confusing and mysterious.
>>
>> Yeah, I thought about having per-directory defaults too. There are some
>> nice things about that. For example, the editing tests mostly want something
>> in between dumpAsText and a render dump. It would be great if we could
>> default those to dumpAsMarkup
>> (https://bugs.webkit.org/show_bug.cgi?id=26501).
>>
>> <3 dumpAsMarkup
>
> In the webkit-meeting discussion about reducing platform-specific tests,
> Adam brought up the idea that we could have a file that lists all the tests
> that default to dumpAsRenderTree, move the default over to dumpAsText and
> then gradually mark tests appropriately as dumpAsRenderTree and remove them
> from the list.
> We might want to leave some directories as defaulting to dumpAsRenderTree as
> they're tests that we imported from elsewhere (e.g. the CSS2.1 tests). We
> can annotate the file appropriately listing just the directories. So,
> eventually, that file will be a few lines.
> I think moving to just having dumpAsText be the default is better than
> having per-directory defaults.
> How does that sound?

I worry that making dumpAsText the default may lead to somewhat
counter-intuitive tests being written. To echo what Maciej seemed to
say earlier, you can write a rendertree test or a pixel test using
vanilla HTML and CSS, but a text-only test almost by definition
requires some amount of JS in the test. At which point, inserting a
dumpAsText() call is no big deal and clearly indicates the intent.

However, having to add DontDumpAsText() as the only script in the test
seems a little silly.

-- Dirk