[webkit-dev] Please don't leave entries for rebaseline in TestExpectation files

[Ryosuke Niwa]

On Thu, Mar 21, 2013 at 1:31 AM, Robert Hogan <lists at roberthogan.net> wrote:

> On Thursday, 21 March 2013, Ryosuke Niwa wrote:
>
>> I used to pull results from the bots where possible but creating
>>> inconsistency between png/text results is not good.
>>>
>>
>> It is unfortunate but it's much better than losing the complete test
>> coverage.
>>
>
> If that's the case then I'm happy to land whatever garden-o-matic pulls in
> or I can sweep from the bots, even if it means that png results for Mac,
> Qt, et al. go bad as a result.
>
> I guess we will always have ports whose bots do not run pixel tests so if
> those ports are happy to live with the downsides of doing that then there
> really is no obstacle to authors owning the job of updating the baselines
> for all ports when they land a change.
>
> IMHO ports who don't run pixel tests would be better off deleting any png
> results they have in the tree. Is there a reason Mac hasn't done that?
> Don't you get lots of failures when you run pixel tests locally?
>

Yes, but I'd argue that it's better than losing the test coverage.

By the way, we can easily address this problem by always generating pixel
results for unexpectedly failing tests. Namely, we can force --pixel when
we're retrying tests.

- R. Niwa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20130321/cc768566/attachment.html>