<br><div class="gmail_quote">On Thu, Mar 21, 2013 at 5:40 PM, Ryosuke Niwa <span dir="ltr"><<a href="mailto:rniwa@webkit.org" target="_blank">rniwa@webkit.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On Thu, Mar 21, 2013 at 4:36 PM, Glenn Adams <span dir="ltr"><<a href="mailto:glenn@skynav.com" target="_blank">glenn@skynav.com</a>></span> wrote:<br></div><div class="gmail_quote"><div class="im">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br><div class="gmail_quote"><div>On Thu, Mar 21, 2013 at 5:10 PM, Ryosuke Niwa <span dir="ltr"><<a href="mailto:rniwa@webkit.org" target="_blank">rniwa@webkit.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>On Thu, Mar 21, 2013 at 4:02 PM, Glenn Adams <span dir="ltr"><<a href="mailto:glenn@skynav.com" target="_blank">glenn@skynav.com</a>></span> wrote:</div><div class="gmail_quote"><div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><div>On Thu, Mar 21, 2013 at 12:49 PM, Ryosuke Niwa <span dir="ltr"><<a href="mailto:rniwa@webkit.org" target="_blank">rniwa@webkit.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>Lately, I've encountering changesets that only add lines to TestExpectations and then never baseline tests for any platform.</div></blockquote><div><br></div></div><div>This (never rebaseline tests for any platform in that changeset) may not be possible depending on circumstances. For example, I do my dev work on MBP Retina, which produces different baselines than the platforms used for mac test bots. As a result, I sometimes have no choice but to land a changeset without a new baseline and then use garden-o-matic after-the-fact to land a new baseline.</div>
</div></blockquote><div><br></div></div><div>Then how are you verifying that your patch is correct? How are reviewers supposed to review such a patch?</div><div><br></div><div>Uploading a rendering engine patch without first verifying that tests are still passing and new tests are generating results as expected sounds like a bad idea to me.</div>
</div></blockquote><div><br></div></div><div>Of course I am verifying that tests are still passing and that new tests are generating results as expected. I do this by running NRWT without the patch, running it with the patch, then comparing results. This generally allows me to see new failures, which I manually review to determine the nature of the difference. If the difference is simply minor pixel positioning deltas (in text or image output), then I operate on the tentative hypothesis that a rebaseline is needed for the given test, unless I happen to be making a change that could be attributed to cause the delta.</div>
</div></blockquote><div><br></div></div><div>That sounds like a dangerous assumption to make. Due to the DPI differences, things like anti-aliasing will behave quite differently on Retina MBP.</div></div></blockquote><div>
<br></div><div>A hypothesis is not an assumption. It needs to be tested further. Be sure I am not making any unwarranted assumption.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><div><br></div><div>In general, I don't recommend people running and relying on layout tests on Retina MBP especially if you work on the rendering engine at this time.</div></div></blockquote>
<div><br></div><div>That's my platform, so I have to manage with it.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote"><div class="im">
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote"><div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote"><div>Or do you have an alternative in mind that would work in my case? Note that it really isn't practical for me to ask another dev to build my patch before landing in order to provide me a new baseline that i can add to the patch.</div>
</div></blockquote><div><br></div></div><div>As I've announced on another thread, EWS now uploads actual results on Bugzilla so this shouldn't be an issue anymore.</div></div></blockquote><div><br></div></div><div>
But EWS is only testing on a couple of platform/ports yes? Presumably this will still impact qt/gtk/efl and perhaps others where EWS is just building and not testing?</div></div></blockquote><div><br></div></div><div>That's fine. I'm asking that you include rebaselines for at least one platform in the case that wasn't clear.</div>
</div></blockquote><div><br></div><div>I will if possible, but I'm just saying it may not be possible on the initial landing in all cases. If it isn't then I expect to have to add rebaseline expectations and then take responsibility for following up on them ASAP.</div>
<div> </div></div>