[webkit-dev] Simplifying syntax in test_expectations.txt (bug 86691)

[Dirk Pranke]

On Thu, May 17, 2012 at 12:48 PM, Dirk Pranke <dpranke at chromium.org> wrote:
> On Thu, May 17, 2012 at 1:06 AM, Maciej Stachowiak <mjs at apple.com> wrote:
>> What does the build configuration info do? Does it apply the line to only those configurations? If that is the case, it does seem potentially different in kind, though maybe also better expressed by being able to combine multiple test_expectations files fro different platform/ directories.
>>
>
> I am planning to add support for cascading files to NRWT shortly (it's
> mostly there now, but I need to add some better diagnostics and
> reimplement the code to actually use the logic), but I would not
> suggest eliminating the build configuration in favor of multiple
> files.
>
> First, you can use build configurations to distinguish between debug
> and release failures, which is very useful.
>
> Second, if you are maintaining multiple platforms and actively trying
> to keep the tree green, I think it is much easier to add one test with
> multiple configurations to one file than to update the same test in
> multiple files.
>
> I think having multiple files makes sense for when expectations
> actually cross organizational boundaries, e.g., we have a set of
> failures that apply to all ports, and a set of failures that are
> implementation specific (e.g., to all chromium ports, or a subset of
> the qt ports). I think it makes sense to organize the files to
> minimize merge conflicts, but at least in chromium's case, we don't
> typically get merge conflicts from platform-specific failures, we get
> them because people are adding test-specific failures.
>

Oh, I should have added that there can be an advantage to having
platform/port/configuration-specific files for long-lived entries (the
WONTFIX lines), since it seems that those can tend to be
configuration-specific and and it makes sense to have all of them in
one place.