[webkit-dev] Best practices for landing new/changed layout test expectations?
dpranke at google.com
Mon Feb 25 13:39:20 PST 2013
On Mon, Feb 25, 2013 at 1:33 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:
> On Mon, Feb 25, 2013 at 1:29 PM, Dirk Pranke <dpranke at google.com> wrote:
>> On Mon, Feb 25, 2013 at 1:22 PM, Ryosuke Niwa <rniwa at webkit.org> wrote:
>> > On Mon, Feb 25, 2013 at 12:52 PM, Dirk Pranke <dpranke at google.com>
>> > wrote:
>> >> > I remember some discussion of a [rebaseline] keyword in
>> >> > TestExpectations,
>> >> > but I'm not sure that ever made it in?
>> >> The [ NeedsRebaseline ] enhancement is, as of yet, unimplemented and
>> >> unclaimed . It shouldn't be too hard for someone to try it if
>> >> they're looking for a reason to explore the NRWT code :)
>> > I object to adding such a thing. People add and forget about these
>> > entries
>> > way too often:
>> The rationale for adding the keyword was precisely this ... it would
>> make it easier to programmatically identify tests needing updated
>> baselines and alert people :).
>> Do you have an alternative you'd suggest for keeping people from
>> forgetting (particularly in the case where we have an expected failure
>> waiting on a bug fix)?
> Just turn bots red. People maintaining bots will notice and rebaseline them
> as needed.
If there's a significant time lapse between failing test and bug fix,
it seems like a bad idea to leave the tree red. Are you suggesting we
should land a "failling" baseline in the meantime?
More information about the webkit-dev