[webkit-dev] handling failing tests (test_expectations, Skipped files, etc.)

[Ryosuke Niwa]

On Tue, Apr 10, 2012 at 12:29 PM, Ojan Vafai <ojan at chromium.org> wrote:

> I don't think we can come up with a hard and fast rule given current
> tooling. In a theoretical future world in which it's easy to get expected
> results off the EWS bots (or some other infrastructure), it would be
> reasonable to expect people to incorporate the correct expected results for
> any EWS-having ports before committing the patch. I expect we'd all agree
> that would be better than turning the bots red or adding to
> test_expectations.txt/Skipped files.
>
> In the current world, it's a judgement call. If I expect a patch to need a
> lot of platform-specific baselines, I'll make sure to commit it at a time
> when I have hours to spare to cleanup any failures or, if I can't stick
> around for the bots to cycle, I'll add it to test_expectations.txt
> appropriately.
>
> Both approaches have nasty tradeoffs. It is probably worth writing up a
> wiki page outlining these two options and explaining why you might do one
> or the other for people new to the project, but I don't see benefit in
> trying to pick a hard rule that everyone must follow.
>

I agree. Both approaches have pros and cons. Will be good to document on
wiki as you say.

- Ryosuke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120410/a2aaff49/attachment.html>