[webkit-dev] handling failing tests (test_expectations, Skipped files, etc.)

[Ryosuke Niwa]

On Mon, Apr 9, 2012 at 2:56 PM, Dana Jansens <danakj at chromium.org> wrote:

> On Mon, Apr 9, 2012 at 5:42 PM, Dirk Pranke <dpranke at chromium.org> wrote:
>
>> 3) Don't use test_expectations.txt to suppress failures across a
>> single cycle of the bot, just so you can gather updated baselines
>> without the tree going red. While it might seem that you're doing tree
>> maintainers a favor, in my experience this just makes things confusing
>> and it's better to let the tree go red until you can actually
>> rebaseline things. It's too easy to add a suppression "for now" and
>> then forget about it.
>>
>
> IIUC, this is not possible if you are landing the change with the commit
> queue, as it will see the failing tests and CQ-? Also, I've been told
> explicitly by reviewers to mark the test fails with a bug filed against it,
> rather than commit something that breaks the tree. Would you see this as
> acceptable behaviour given the bug filed against it and in a comment in
> test_expectations?
>

In my opinion, if you're making changes such that it requires test
platform-specific rebaselines, you shouldn't be landing it via commit
queue. Instead, ask your reviewer or other committer to land your change,
and then coordinate with her or him to do necessary rebaselines on each
port.

- Ryosuke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120409/635a6876/attachment.html>