[webkit-dev] Skipping Flakey Tests

Maciej Stachowiak mjs at apple.com
Tue Dec 22 20:11:01 PST 2009

On Dec 22, 2009, at 10:31 AM, Darin Adler wrote:

> On Dec 21, 2009, at 6:14 PM, Dirk Pranke wrote:
>> Given all that, Darin, what were you suggesting when you said  
>> "Let's fix that"?
> Lets add a feature so something in the tests tree can indicate a  
> Chromium Windows result should be the base result, rather than the  
> platform/win one. We can debate exactly how it should work, but lets  
> come up with a design and do it.

One possibility is to have three windows-related result directories,  
win/, win-cg/ and win-chromium/. We'd use the latter two for test  
results that are different between the CoreGraphics/CFNetwork/Apple  
port vs. the Chromium/Google port. Any results that should be common  
to all windows-based platforms.

Note: As far as I can tell, this problem is all about granularity of  
applying the expected results, not about whether we check in known  
failing results instead of out-of-band FAIL expectations. The latter  
may or may not be a good idea(*), but it seems independent of the  
original problem. Some of Dirk and Peter's messages seemed to conflate  
these two points.


* - I tend to think we should track expected failure results, for the  
reasons Darin cited and also because tests that "fail" already can  
still catch further regressions.

More information about the webkit-dev mailing list