[webkit-dev] Please don't leave entries for rebaseline in TestExpectation files

[Ryosuke Niwa]

On Thu, Mar 21, 2013 at 1:57 AM, Ryosuke Niwa <rniwa at webkit.org> wrote:

> On Thu, Mar 21, 2013 at 1:31 AM, Robert Hogan <lists at roberthogan.net>wrote:
>
>> On Thursday, 21 March 2013, Ryosuke Niwa wrote:
>>
>>> I used to pull results from the bots where possible but creating
>>>> inconsistency between png/text results is not good.
>>>>
>>>
>>> It is unfortunate but it's much better than losing the complete test
>>> coverage.
>>>
>>
>> If that's the case then I'm happy to land whatever garden-o-matic pulls
>> in or I can sweep from the bots, even if it means that png results for Mac,
>> Qt, et al. go bad as a result.
>>
>> I guess we will always have ports whose bots do not run pixel tests so if
>> those ports are happy to live with the downsides of doing that then there
>> really is no obstacle to authors owning the job of updating the baselines
>> for all ports when they land a change.
>>
>> IMHO ports who don't run pixel tests would be better off deleting any png
>> results they have in the tree. Is there a reason Mac hasn't done that?
>> Don't you get lots of failures when you run pixel tests locally?
>>
>
> Yes, but I'd argue that it's better than losing the test coverage.
>
> By the way, we can easily address this problem by always generating pixel
> results for unexpectedly failing tests. Namely, we can force --pixel when
> we're retrying tests.
>

I've posted a patch on https://bugs.webkit.org/show_bug.cgi?id=112898 to
implement this. Once that patch is landed, you can grab pixel results from
any EWS bots and bots on build.webkit.org.

- R. Niwa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20130322/dfc65f20/attachment.html>