[webkit-dev] Why are some layout tests renamed with a suffix of -disabled?

[Dan Bernstein]

On Nov 25, 2009, at 10:38 AM, Darin Fisher wrote:

> On Wed, Nov 25, 2009 at 10:35 AM, Dan Bernstein <mitz at apple.com> wrote:
> 
> On Nov 25, 2009, at 9:55 AM, Darin Fisher wrote:
> 
> > Why are some layout tests renamed with a suffix of -disabled?  Why aren't such tests simply added to the skipped list?  Is -disabled simply the old way?
> 
> Usually a test is disabled, with a bug filed to re-enable it, when a WebKit bug makes it impossible to run the test (e.g. it crashes DumpRenderTree) or makes the test produce different results on each run (this can also be a bug in the test). The skipped lists are platform-specific, so they are not a good way to deal with such situations.
> 
> My concern is that some tests may pass on some ports but not others.

For such cases, the skipped list is the way to go.

> If the tests are -disabled, then it prevents them from being run on ports where the tests function properly.

I see now that I didn’t make myself clear. By “WebKit bug” above I meant a bug that affects all ports and platforms (maybe I should have said “a core WebKit bug” or “a WebCore bug” :-) ).

> Since skipped lists prevent the test from being executed, doesn't it solve the problem of disabling a test for ports that can't handle it?  What am I missing? :-)

The “tests that crash/hang/behave unpredictably on all ports” part.

There aren’t many -disabled tests, and I think it’s a good idea to audit them and make sure that (a) they can’t be re-enabled (b) they can’t just be skipped on some platforms and run on others and (c) there’s a bug filed on re-enabling them (or fixing the reason why they can’t be enabled).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20091125/5e354a41/attachment.html>