[Webkit-unassigned] [Bug 86796] Change a bunch of things in the test_expectations.txt format

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=86796





--- Comment #2 from Ryosuke Niwa <rniwa at webkit.org>  2012-05-17 19:42:30 PST ---
(In reply to comment #1)
> (In reply to comment #0)
> > As per the result of https://lists.webkit.org/pipermail/webkit-dev/2012-May/020674.html, we've agree to make the following changes to the test_expectations.txt format.
> 
> Not to be a huge pain, but I'm not sure that there was "agreement" on this; I'd kinda like to see Maciej and/or Darin weigh in.

It's my understanding that nobody has objected to these changes so far so we're on rough consensus. We don't necessarily ask everyone to explicitly say they concur since WebKit isn't really a standard body but I think that's okay.

> For the record, though ...
> > 
> > 1. Make SLOW, SKIP and WONTFIX outcomes instead of modifiers.
> 
> Does SLOW imply PASS? We certainly have slow failures today, and if it doesn't imply pass, but only keeps its current semantics, it seems like it complicates the implementation (SLOW is now "special"). We could also just get rid of SLOW.

I think we're not changing the semantics of SLOW. So if we have
wkbug.com/12345 : WIN MAC test.html : SLOW IMAGE TEXT
then that means the test is slow, and may have image or text failures.

> > 4. WONTFIX tests are either skipped or we ignore their output as long as they don't crash (slight preference for skipping them, but not enough to argue over it).
> 
> We should skip the WONTFIX tests. I don't think we should ignore output if we're going to run tests. 

Yes.

> > 5. Use the same character as a separator before and after the test (":").
> 
> I'm not sure this is an improvement. If we were going to standardize on the same delimiter I would be open to using one that doesn't require a shift key as per request from rniwa. Possible candidates: "-", ";", ",", "=" (which I would prefer in that order, I think; frankly, I only really like "-" here, and I'm not sure I like any of them more than the existing ":"/"=" combo).

I would prefer ";" or "," because they are used as delimiters in English sentences and many programming languages.

Or maybe [ ] as in:

webkit.org/12345 WIN MAC DEBUG [ animations/stop-animation-on-suspend.html ] CRASH TEXT PASS

This syntax begs for allowing multiple tests inside [ and ] though:

webkit.org/12345 WIN MAC DEBUG [ animations/my-test.html, animations/another-test.html ] CRASH TEXT PASS

It's probably another controversial change... so I'm not going to formally propose it.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.