[webkit-dev] Please don't leave entries for rebaseline in TestExpectation files

[Robert Hogan]

On Wednesday, 20 March 2013, Ryosuke Niwa wrote:

> Please don't add lines to TestExpectations saying that they just need
> rebaselines and then leave.
>
>
OK. That means I will have to pull the new results from the bots, which is
fine - but in the case of the Mac port (and any other bot that does not run
pixel tests) the result will be that trunk will get fresh text results but
retain stale png results.

If that is OK then you need to publish that information somewhere as I
suspect I'm not the only contributor who has hesitated to make Mac's test
results inconsistent.

That would reduce the test coverage we have, and effectively disables the
> test. If you're adding those entires, please be sure to remove them
> ASAP. Better yet, don't add them unless you have to rebaseline hundreds of
> tests. It's not acceptable to leave those entries in TestExpectations for
> days.
>

 We've batted back and forth on this list for at least a year on the
correct approach for landing and rebaselining. My approach is to land
results for the platform that I build, suppress tests that require
rebaselining on other platforms, and open a bug so sheriffs can
add/rebaseline results as appropriate.

My impression from recent discussion on this topic was that this was the
way that worked best for everybody.I used to pull results from the bots
where possible but creating inconsistency between png/text results is not
good.

Presumably this will be discussed at the contributors' meeting - it would
be good to make sure that all the relevant people required for consensus on
this topic can attend the discussion and settle this once and for all!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20130321/4e709a19/attachment.html>