[webkit-dev] Process for making changes that affect layout test results

[Tony Payne]

Some of these test result changes turn out to be the effect of incorrect
test cases.

For example, compositing/color-matching/image-color-matching.html compares
an image without a color profile with an image tagged with what it calls a
"default RGB" profile. However, this "default RGB" profile is not sRGB, so
the test is wrongly[1] assuming these images should be displayed the same.
I'd like to replace all of the resources containing random RGB profiles
with sRGB profiles instead.

1) Is this a good idea? Is there a reason why these image resources have
non-standard profiles in them?
2) How do I coordinate this, as changing the image resources seems like it
will affect all the ports?

Tony

[1] http://www.w3.org/Graphics/Color/sRGB.html

On Wed, Apr 11, 2012 at 3:36 PM, Dirk Pranke <dpranke at chromium.org> wrote:

> use run-webkit-tests (or attempt to pull the results from the trybots
> by hand). You cannot pull new baselines from the try bots using
> rebaseline.py.
>
> -- Dirk
>
> On Wed, Apr 11, 2012 at 3:26 PM, Tony Payne <tpayne at chromium.org> wrote:
> > Is the best way to do that to use run_webkit_tests to generate a new
> > baseline or does Tools/Scripts/webkitpy/tool/commands/rebaseline.py
> support
> > pulling new baselines from the trybots?
> >
> > Thanks,
> > Tony
> >
> > On Wed, Apr 11, 2012 at 2:13 PM, Ojan Vafai <ojan at chromium.org> wrote:
> >>
> >> Typically, if you're working on Chromium Linux or Win, you'd include the
> >> new expected results for that platform in your initial
> commit/code-review as
> >> well.
> >>
> >> On Wed, Apr 11, 2012 at 2:09 PM, Tony Payne <tpayne at chromium.org>
> wrote:
> >>>
> >>> All code I'm changing is inside of #if PLATFORM(CHROMIUM) blocks.
> >>>
> >>> Thanks for the quick answer.
> >>>
> >>> Tony
> >>>
> >>>
> >>> On Wed, Apr 11, 2012 at 2:03 PM, Ryosuke Niwa <rniwa at webkit.org>
> wrote:
> >>>>
> >>>> On Wed, Apr 11, 2012 at 1:57 PM, Tony Payne <tpayne at chromium.org>
> wrote:
> >>>>>
> >>>>> Given the recent discussion on test_expectations.txt, perhaps the
> >>>>> answer to my question is still up in the air.
> >>>>>
> >>>>> I'm working on a change that I expect to require changing the
> >>>>> expectations for about 75 tests on chromium win and
> >>>>> linux. https://trac.webkit.org/wiki/Rebaseline seems to only cover
> the
> >>>>> gardening work to rebaseline after the commit. I cannot find any
> wiki pages
> >>>>> that describe what the original author is expected to do when making
> visual
> >>>>> changes. Should I attempt to rebaseline manually? Should I mark the
> tests as
> >>>>> failing? Should I just check in and let the bots go red?
> >>>>
> >>>>
> >>>> Just land the patch and rebaseline the tests. Please also coordinate
> >>>> with Chromium port's WebKit gardener when landing this patch.
> >>>>
> >>>> Also, does this patch only affect Chromium Windows and Linux, and not
> >>>> GTK, Qt, Windows, etc...? If the answer is no, and will affect other
> >>>> non-Chromium ports, then you're also responsible for rebaselining or
> >>>> coordinating with other ports to make sure you don't break tests on
> their
> >>>> ports as well.
> >>>>
> >>>> - Ryosuke
> >>>>
> >>>
> >>>
> >>> _______________________________________________
> >>> webkit-dev mailing list
> >>> webkit-dev at lists.webkit.org
> >>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
> >>>
> >>
> >
> >
> > _______________________________________________
> > webkit-dev mailing list
> > webkit-dev at lists.webkit.org
> > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20120411/34ca7126/attachment.html>