[webkit-dev] A not-quite baked proposal for moving pixel baselines out of trunk

[Adam Barth]

On Mon, Sep 12, 2011 at 4:11 PM, Jarred Nicholls <jarred at sencha.com> wrote:
> On Mon, Sep 12, 2011 at 1:48 PM, Adam Barth <abarth at webkit.org> wrote:
>>
>> Leandro's recent message reminded me that there was some discussion on
>> this list about the burden caused by storing and maintaining pixel
>> baselines for an ever-growing list of configurations.  I've been
>> working on a proposal for moving pixel baselines to a "branch" so that
>> they don't bother most folks.
>>
>> Unfortunately, I haven't been able to work out all the details yet.
>> Here's a somewhat rough, work-in-progress proposal.  If you like this
>> approach, I can spend more time trying to solve the remaining
>> problems.
>>
>> == Motivation ==
>>
>> Maintaining pixel test results in svn.webkit.org for the various
>> Chromium configuration imposes a number of costs on the WebKit
>> community:
>>
>> All members of the community need to store these results locally when
>> they create a checkout of WebKit.
>> Updating the pixel results creates churn in the project that increases
>> the time required to update a working copy and makes it more difficult
>> to understand what is actually changing in the project.
>>
>> As the Chromium port grows in complexity, there is pressure from the
>> Chromium project to expand the number of test configurations,
>> increasing these costs on the WebKit community.
>>
>> == Proposal (Work-in-progress) ==
>>
>> One option is to stop running pixel test altogether, but that
>> decreases test coverage and costs correctness.  Another possibility is
>> to store the pixel results in a location where they are largely
>> invisible to the rest of the WebKit community.  For example, we could
>> store the pixel results for chromium-linux the following location:
>>
>> http://svn.webkit.org/repository/webkit/PixelResults/chromium-linux
>>
>> Chromium contributors could then use gclient and DEPS to map these
>> results into their working copies and non-chromium contributors could
>> safely ignore any changes in the PixelResults “branch”.
>>
>> FIXME: What about non-platform specific results?
>>
>> == Problems ==
>>
>> The main reason this proposal is half-baked is I haven't quite been
>> able to work out how folks can do some common operations:
>>
>> 1) Land patch with expected image failures (+ revert)
>> 2) Land patch with new image baselines (+ revert)
>> 3) Land new baselines
>>
>> Because the pixel results and trunk are in the same SVN repo, we
>> should be able to do these things atomically, but dealing with the
>> sparse checkout is kind of tricky.
>>
>> Another big question mark is how this would work for contributors who
>> use git.  When git mirrors an SVN repro, it only mirrors "trunk", so
>> the PixelResults directory wouldn't exist in the git version of the
>> repository.
>
> git-svn would be the fallback - I'd rather take razor blades to my eyes.
>  But this is a noble effort in general, thanks.
> Do we need to do anything to alleviate history in the git mirror?  I'd love
> to clone 25MB and not 2.1GB.

I'm not a git expert, so I'm not sure if there's any way to repair the
sins of the past (e.g., with filter-branch) while still keeping
git-svn working.

Adam