[webkit-dev] Process for making changes that affect layout test results

[Dirk Pranke]

On Wed, Apr 11, 2012 at 4:56 PM, Tony Payne <tpayne at chromium.org> wrote:
> Some of these test result changes turn out to be the effect of incorrect
> test cases.
>
> For example, compositing/color-matching/image-color-matching.html compares
> an image without a color profile with an image tagged with what it calls a
> "default RGB" profile. However, this "default RGB" profile is not sRGB, so
> the test is wrongly[1] assuming these images should be displayed the same.
> I'd like to replace all of the resources containing random RGB profiles with
> sRGB profiles instead.
>
> 1) Is this a good idea? Is there a reason why these image resources have
> non-standard profiles in them?

I don't know the answer to this, but hopefully someone else will.

> 2) How do I coordinate this, as changing the image resources seems like it
> will affect all the ports?

To the extent you can notify other port maintainers, I'm sure they
would appreciate it. Otherwise, treat it the same way you would treat
any other change ... It will just be more painful :).

-- Dirk
>
> Tony
>
> [1] http://www.w3.org/Graphics/Color/sRGB.html
>
> On Wed, Apr 11, 2012 at 3:36 PM, Dirk Pranke <dpranke at chromium.org> wrote:
>>
>> use run-webkit-tests (or attempt to pull the results from the trybots
>> by hand). You cannot pull new baselines from the try bots using
>> rebaseline.py.
>>
>> -- Dirk
>>
>> On Wed, Apr 11, 2012 at 3:26 PM, Tony Payne <tpayne at chromium.org> wrote:
>> > Is the best way to do that to use run_webkit_tests to generate a new
>> > baseline or does Tools/Scripts/webkitpy/tool/commands/rebaseline.py
>> > support
>> > pulling new baselines from the trybots?
>> >
>> > Thanks,
>> > Tony
>> >
>> > On Wed, Apr 11, 2012 at 2:13 PM, Ojan Vafai <ojan at chromium.org> wrote:
>> >>
>> >> Typically, if you're working on Chromium Linux or Win, you'd include
>> >> the
>> >> new expected results for that platform in your initial
>> >> commit/code-review as
>> >> well.
>> >>
>> >> On Wed, Apr 11, 2012 at 2:09 PM, Tony Payne <tpayne at chromium.org>
>> >> wrote:
>> >>>
>> >>> All code I'm changing is inside of #if PLATFORM(CHROMIUM) blocks.
>> >>>
>> >>> Thanks for the quick answer.
>> >>>
>> >>> Tony
>> >>>
>> >>>
>> >>> On Wed, Apr 11, 2012 at 2:03 PM, Ryosuke Niwa <rniwa at webkit.org>
>> >>> wrote:
>> >>>>
>> >>>> On Wed, Apr 11, 2012 at 1:57 PM, Tony Payne <tpayne at chromium.org>
>> >>>> wrote:
>> >>>>>
>> >>>>> Given the recent discussion on test_expectations.txt, perhaps the
>> >>>>> answer to my question is still up in the air.
>> >>>>>
>> >>>>> I'm working on a change that I expect to require changing the
>> >>>>> expectations for about 75 tests on chromium win and
>> >>>>> linux. https://trac.webkit.org/wiki/Rebaseline seems to only cover
>> >>>>> the
>> >>>>> gardening work to rebaseline after the commit. I cannot find any
>> >>>>> wiki pages
>> >>>>> that describe what the original author is expected to do when making
>> >>>>> visual
>> >>>>> changes. Should I attempt to rebaseline manually? Should I mark the
>> >>>>> tests as
>> >>>>> failing? Should I just check in and let the bots go red?
>> >>>>
>> >>>>
>> >>>> Just land the patch and rebaseline the tests. Please also coordinate
>> >>>> with Chromium port's WebKit gardener when landing this patch.
>> >>>>
>> >>>> Also, does this patch only affect Chromium Windows and Linux, and not
>> >>>> GTK, Qt, Windows, etc...? If the answer is no, and will affect other
>> >>>> non-Chromium ports, then you're also responsible for rebaselining or
>> >>>> coordinating with other ports to make sure you don't break tests on
>> >>>> their
>> >>>> ports as well.
>> >>>>
>> >>>> - Ryosuke
>> >>>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> webkit-dev mailing list
>> >>> webkit-dev at lists.webkit.org
>> >>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>> >>>
>> >>
>> >
>> >
>> > _______________________________________________
>> > webkit-dev mailing list
>> > webkit-dev at lists.webkit.org
>> > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>> >
>
>