[webkit-dev] A simpler proposal for handling failing tests WAS: A proposal for handling "failing" layout tests and TestExpectations

[Filip Pizlo]

+1, contingent upon the following: are we agreeing that all current uses of TEXT, IMAGE, and so forth in TestExpectations should be in the *very near term* following Dirk's change be turned into -failing files?

-Filip


On Aug 17, 2012, at 5:01 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:

> On Fri, Aug 17, 2012 at 4:55 PM, Ojan Vafai <ojan at chromium.org> wrote:
> Asserting a test case is 100% correct is nearly impossible for a large percentage of tests. The main advantage it gives us is the ability to have -expected mean "unsure".
> 
> Lets instead only add -failing (i.e. no -passing). Leaving -expected to mean roughly what it does today to Chromium folk (roughly, as best we can tell this test is passing). -failing means it's *probably* an incorrect result but needs an expert to look at it to either mark it correct (i.e. rename it to -expected) or figure out how the root cause of the bug.
> 
> This actually matches exactly what Chromium gardeners do today, except instead of putting a line in TestExpectations/Skipped to look at later, they checkin the -failing file to look at later, which has all the advantages Dirk listed in the other thread.
> 
> I'm much more comfortable with this proposal.
> 
> - Ryosuke
> 
> _______________________________________________
> webkit-dev mailing list
> webkit-dev at lists.webkit.org
> http://lists.webkit.org/mailman/listinfo/webkit-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120817/3a652285/attachment.html>