[webkit-dev] Why are some layout tests renamed with a suffix of -disabled?

[Darin Fisher]

On Wed, Nov 25, 2009 at 10:44 AM, Dan Bernstein <mitz at apple.com> wrote:

>
> On Nov 25, 2009, at 10:38 AM, Darin Fisher wrote:
>
> On Wed, Nov 25, 2009 at 10:35 AM, Dan Bernstein <mitz at apple.com> wrote:
>
>>
>> On Nov 25, 2009, at 9:55 AM, Darin Fisher wrote:
>>
>> > Why are some layout tests renamed with a suffix of -disabled?  Why
>> aren't such tests simply added to the skipped list?  Is -disabled simply the
>> old way?
>>
>> Usually a test is disabled, with a bug filed to re-enable it, when a
>> WebKit bug makes it impossible to run the test (e.g. it crashes
>> DumpRenderTree) or makes the test produce different results on each run
>> (this can also be a bug in the test). The skipped lists are
>> platform-specific, so they are not a good way to deal with such situations.
>
>
> My concern is that some tests may pass on some ports but not others.
>
>
> For such cases, the skipped list is the way to go.
>
> If the tests are -disabled, then it prevents them from being run on ports
> where the tests function properly.
>
>
> I see now that I didn’t make myself clear. By “WebKit bug” above I meant a
> bug that affects all ports and platforms (maybe I should have said “a core
> WebKit bug” or “a WebCore bug” :-) ).
>
> Since skipped lists prevent the test from being executed, doesn't it solve
> the problem of disabling a test for ports that can't handle it?  What am I
> missing? :-)
>
>
> The “tests that crash/hang/behave unpredictably on all ports” part.
>
> There aren’t many -disabled tests, and I think it’s a good idea to audit
> them and make sure that (a) they can’t be re-enabled (b) they can’t just be
> skipped on some platforms and run on others and (c) there’s a bug filed on
> re-enabling them (or fixing the reason why they can’t be enabled).
>


This makes sense now.  Thanks for clarifying.

-Darin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20091125/f584f818/attachment.html>