[Webkit-unassigned] [Bug 65834] NRWT should support cascading expectations

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=65834





--- Comment #4 from Balazs Kelemen <kbalazs at webkit.org>  2011-08-09 01:41:31 PST ---
(In reply to comment #3)
> We don't have tree fallback paths now, I doubt we'll easily be able to get rid of them, and changing the teset expectations shouldn't change the fallback paths, regardless (they're different concepts).

As I know with ORWT the path of cascading the results was the same as the fallback paths. I think this is the only logical way of this.

> [ Unless of course we fundamentally want to change some aspect of how test expectations work ].
> 
> That said, the only way to currently share expectations is to share the same expectations.txt file.
> 
> Given that Chromium ports currently use the expectations file to track that the expectation is "expected to fail" and hence are using the expectations as "correctness" tests, and the other ports check in "failing" expectations and hence use the expectations as "regression" tests, we almost certainly don't want to share Chromium- and non-Chromium expectations.txt files. Besides, the Chromium file is is already way too big and churns too often.
> 

I think the cascading has nothing to do with the way how we interpret the results. Cascading is useful for both regression and correctness tests.
The point is that a port has subports (like qt and qt-linux) and the subport
has some metric differences but a great amount of expectation can be shared
across subports.

> It's not clear to me that we need more than a "generic" set of expectations that applies to all ports in addition to the the "port-specific" expectations. I would be tempted to prototype that first and see if that got us what we wanted. 

I don't think we need a generic set of expectations. On the contrary, we need less generic expectations for subports.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.