[Webkit-unassigned] [Bug 105860] Tests with WontFix expectation are (indirectly) skipped

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=105860





--- Comment #3 from Dirk Pranke <dpranke at chromium.org>  2012-12-29 14:44:26 PST ---
Note that NRWT actually used to do what you wanted (i.e., WontFix did not imply Skip). There was a fair amount of debate over the years as to whether or not running the WontFix-but-not-Skip tests was at all useful for catching the sort of crashes you mention. The closest anyone came was the belief that it did catch a few things when the port was very new (i.e., we were still bringing it up and crashes were not uncommon).

Eventually we mostly agreed w/ Eric; the cost of running tests we didn't care about seemed excessive, especially given that we had so much other coverage, and so we changed what WontFix meant (IIRC, Apple also wanted WontFix to imply Skip, although they probably would've been happy w/ just Skip).

But, not every port is Chromium - I could see an argument that other, especially newer, ports might want or need better coverage from the suite. I don't think we're likely to change back to the old behavior generally, but I could possibly be convinced to make this a port-specific configurable parameter whether things are skipped or not. Even then, this might be a bad idea since different ports would work differently.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.