[webkit-dev] Best practices for landing new/changed layout test expectations?

[Dirk Pranke]

On Mon, Feb 25, 2013 at 1:33 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:
> On Mon, Feb 25, 2013 at 1:29 PM, Dirk Pranke <dpranke at google.com> wrote:
>>
>> On Mon, Feb 25, 2013 at 1:22 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:
>> > On Mon, Feb 25, 2013 at 12:52 PM, Dirk Pranke <dpranke at google.com>
>> > wrote:
>> >
>> >> > I remember some discussion of a [rebaseline] keyword in
>> >> > TestExpectations,
>> >> > but I'm not sure that ever made it in?
>> >>
>> >> The [ NeedsRebaseline ] enhancement is, as of yet, unimplemented and
>> >> unclaimed [2]. It shouldn't be too hard for someone to try it if
>> >> they're looking for a reason to explore the NRWT code :)
>> >
>> >
>> > I object to adding such a thing. People add and forget about these
>> > entries
>> > way too often:
>> >
>>
>> The rationale for adding the keyword was precisely this ... it would
>> make it easier to programmatically identify tests needing updated
>> baselines and alert people :).
>>
>> Do you have an alternative you'd suggest for keeping people from
>> forgetting (particularly in the case where we have an expected failure
>> waiting on a bug fix)?
>
> Just turn bots red. People maintaining bots will notice and rebaseline them
> as needed.
>

If there's a significant time lapse between failing test and bug fix,
it seems like a bad idea to leave the tree red. Are you suggesting we
should land a "failling" baseline in the meantime?

-- Dirk