[Webkit-unassigned] [Bug 61888] New: nrwt: should skip chromium test expectation overrides on upstream bots

bugzilla-daemon at webkit.org bugzilla-daemon at webkit.org
Wed Jun 1 15:49:43 PDT 2011


https://bugs.webkit.org/show_bug.cgi?id=61888

           Summary: nrwt: should skip chromium test expectation overrides
                    on upstream bots
           Product: WebKit
           Version: 528+ (Nightly build)
          Platform: Unspecified
        OS/Version: Unspecified
            Status: NEW
          Severity: Normal
          Priority: P2
         Component: Tools / Tests
        AssignedTo: webkit-unassigned at lists.webkit.org
        ReportedBy: dpranke at chromium.org
                CC: tony at chromium.org, abarth at webkit.org


If there are test_expectations checked into a downstream Chromium checkout, they are automatically picked up by both the canaries and the DEPS bots. There is therefore no way to test whether the upstream test_expectations.txt themselves are sufficient to keep the tree green (which one might want to do prior to doing a webkit roll).

So, we've decided to modify the code so that the overrides are ignored if the builder_name is set and does not include "(deps)" in the name.

A downside of this change is that a chromium-only committer will no longer be able to fix a failure on the canaries by just inserting a downstream expectation. Hopefully this will not be much of an issue in practice.


Note that there are two ways to implement this change. We could either make the code generically aware of the significance of "(deps)" and skip the overrides generically, or we could push all of this logic inside the chromium code.

There does not appear to be a way to write tests for this that will cover both cases :(

I will post patches that demonstrate both approaches, but the chromium-only approach seems slightly simpler for now.

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.



More information about the webkit-unassigned mailing list