[Webkit-unassigned] [Bug 96718] mac-future ignores fallback test expectations

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=96718





--- Comment #7 from Dirk Pranke <dpranke at chromium.org>  2012-09-14 11:05:57 PST ---
I'm sorry, I'm surely missing something here. 

First, if this discussion is blocking anything important, feel free to have someone r+ this and land it while we try to resolve my confusion; the patch looks fine but I don't want to r+ something I don't understand the need for.

I understand the desire to distinguish between the version that is currently shipping and some internal unreleased version. I believe that in order to run run-webkit-tests with some new set of tests, baselines, and expectations, running with --platform mac-future --additional-platform-dir=X --additional-expectations=Y is all you should need. That will pick up the TestExpectations in platform/mac in addition to the file you've specified on the command line. You can verify this by, e.g., running with --details and checking the expectation for some test listed in mac/TestExpectations, like fast/dom/shadow/content-element-api.html.

The point I'm confused about is your need to modify _generate_all_test_configurations(); while that function is called during run-webkit-tests, it isn't (I don't think) really used for anything, and I don't think adding mac-future to that will have any effect on how run-webkit-tests runs. 

The list of configurations is used mostly (solely?) when using the 'REBASELINE' modifier and rewriting the file when tests no longer fail. So, I could believe that mac-future needs to be in that list if you're trying to make it work with garden-o-matic or webkit-patch rebaseline-expectations. Are you trying to use one of those commands, or are you having some problem with run-webkit-tests that I'm not seeing?

-- 
Configure bugmail: https://bugs.webkit.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.