[webkit-dev] Process for making changes that affect layout test results

[Tony Payne]

On Wed, Apr 11, 2012 at 4:58 PM, Dirk Pranke <dpranke at chromium.org> wrote:

> On Wed, Apr 11, 2012 at 4:56 PM, Tony Payne <tpayne at chromium.org> wrote:
> > Some of these test result changes turn out to be the effect of incorrect
> > test cases.
> >
> > For example, compositing/color-matching/image-color-matching.html
> compares
> > an image without a color profile with an image tagged with what it calls
> a
> > "default RGB" profile. However, this "default RGB" profile is not sRGB,
> so
> > the test is wrongly[1] assuming these images should be displayed the
> same.
> > I'd like to replace all of the resources containing random RGB profiles
> with
> > sRGB profiles instead.
> >
> > 1) Is this a good idea? Is there a reason why these image resources have
> > non-standard profiles in them?
>
> I don't know the answer to this, but hopefully someone else will.


The patch is here:
https://bugs.webkit.org/attachment.cgi?id=136946&action=review

If anyone knows why these images had the color profiles they had or a
reason not to assign them to sRGB, please let me know.


> > 2) How do I coordinate this, as changing the image resources seems like
> it
> > will affect all the ports?
>
> To the extent you can notify other port maintainers, I'm sure they
> would appreciate it. Otherwise, treat it the same way you would treat
> any other change ... It will just be more painful :).
>
> -- Dirk
> >
> > Tony
> >
> > [1] http://www.w3.org/Graphics/Color/sRGB.html
> >
> > On Wed, Apr 11, 2012 at 3:36 PM, Dirk Pranke <dpranke at chromium.org>
> wrote:
> >>
> >> use run-webkit-tests (or attempt to pull the results from the trybots
> >> by hand). You cannot pull new baselines from the try bots using
> >> rebaseline.py.
> >>
> >> -- Dirk
> >>
> >> On Wed, Apr 11, 2012 at 3:26 PM, Tony Payne <tpayne at chromium.org>
> wrote:
> >> > Is the best way to do that to use run_webkit_tests to generate a new
> >> > baseline or does Tools/Scripts/webkitpy/tool/commands/rebaseline.py
> >> > support
> >> > pulling new baselines from the trybots?
> >> >
> >> > Thanks,
> >> > Tony
> >> >
> >> > On Wed, Apr 11, 2012 at 2:13 PM, Ojan Vafai <ojan at chromium.org>
> wrote:
> >> >>
> >> >> Typically, if you're working on Chromium Linux or Win, you'd include
> >> >> the
> >> >> new expected results for that platform in your initial
> >> >> commit/code-review as
> >> >> well.
> >> >>
> >> >> On Wed, Apr 11, 2012 at 2:09 PM, Tony Payne <tpayne at chromium.org>
> >> >> wrote:
> >> >>>
> >> >>> All code I'm changing is inside of #if PLATFORM(CHROMIUM) blocks.
> >> >>>
> >> >>> Thanks for the quick answer.
> >> >>>
> >> >>> Tony
> >> >>>
> >> >>>
> >> >>> On Wed, Apr 11, 2012 at 2:03 PM, Ryosuke Niwa <rniwa at webkit.org>
> >> >>> wrote:
> >> >>>>
> >> >>>> On Wed, Apr 11, 2012 at 1:57 PM, Tony Payne <tpayne at chromium.org>
> >> >>>> wrote:
> >> >>>>>
> >> >>>>> Given the recent discussion on test_expectations.txt, perhaps the
> >> >>>>> answer to my question is still up in the air.
> >> >>>>>
> >> >>>>> I'm working on a change that I expect to require changing the
> >> >>>>> expectations for about 75 tests on chromium win and
> >> >>>>> linux. https://trac.webkit.org/wiki/Rebaseline seems to only
> cover
> >> >>>>> the
> >> >>>>> gardening work to rebaseline after the commit. I cannot find any
> >> >>>>> wiki pages
> >> >>>>> that describe what the original author is expected to do when
> making
> >> >>>>> visual
> >> >>>>> changes. Should I attempt to rebaseline manually? Should I mark
> the
> >> >>>>> tests as
> >> >>>>> failing? Should I just check in and let the bots go red?
> >> >>>>
> >> >>>>
> >> >>>> Just land the patch and rebaseline the tests. Please also
> coordinate
> >> >>>> with Chromium port's WebKit gardener when landing this patch.
> >> >>>>
> >> >>>> Also, does this patch only affect Chromium Windows and Linux, and
> not
> >> >>>> GTK, Qt, Windows, etc...? If the answer is no, and will affect
> other
> >> >>>> non-Chromium ports, then you're also responsible for rebaselining
> or
> >> >>>> coordinating with other ports to make sure you don't break tests on
> >> >>>> their
> >> >>>> ports as well.
> >> >>>>
> >> >>>> - Ryosuke
> >> >>>>
> >> >>>
> >> >>>
> >> >>> _______________________________________________
> >> >>> webkit-dev mailing list
> >> >>> webkit-dev at lists.webkit.org
> >> >>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
> >> >>>
> >> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > webkit-dev mailing list
> >> > webkit-dev at lists.webkit.org
> >> > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
> >> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120412/c2151b79/attachment.html>