[webkit-dev] Faster Git SVN updates

[Eric Seidel]

WARNING:  This setup seems to have a race which can get your
repository into an inconsistent state.  It's happened now 3 times on 2
of our commit bots:
https://bugs.webkit.org/show_bug.cgi?id=50825#c16

Your repository gets into a state where git svn rebase fails to apply
the rebase from upstream.  It's unclear what causes this, but once
your'e in it, you have to:
git rebase --abort
git svn reset KNOWN_GOOD_REVISION
git rebase --hard KNOWN_GOOD_GIT_HASH

I've moved the commit-bots off of this setup and no longer recommend
it for anyone who is committing.

-eric

On Tue, Nov 30, 2010 at 10:59 PM, Eric Seidel <eric at webkit.org> wrote:
> I am of the impression that now any git user need only run:
> git config --replace-all svn-remote.svn.fetch
> trunk:refs/remotes/origin/master
> in their get repo and updates will be faster.  I'm doing so now on the EWS
> bots, and will fix any tooling issues in encounter.
> I think it's time that we consider adding a "git" section
> to http://webkit.org/building/checkout.html.
> -eric
>
> On Tue, Nov 30, 2010 at 2:20 PM, Eric Seidel <eric at webkit.org> wrote:
>>
>> I've now posted a patch to fix update-webkit as well:
>> https://bugs.webkit.org/show_bug.cgi?id=50273
>> Once that lands, I'll move the EWS bots over to using this "new" setup.
>>  Assuming those stay working, we can teach the tools to offer to fix "old"
>> setups.
>> -eric
>>
>> On Mon, Nov 22, 2010 at 4:58 PM, Ojan Vafai <ojan at chromium.org> wrote:
>>>
>>> http://trac.webkit.org/changeset/72575 for having scm.py do the right
>>> thing.
>>>
>>> On Thu, Nov 18, 2010 at 4:11 PM, Eric Seidel <eric at webkit.org> wrote:
>>>>
>>>> OK.  Sounds we should make this setup default.  I'll see if we can't
>>>> educate update-webkit and webkitpy/common/checkout/scm.py about detecting
>>>> this setup and doing the right thing in that case.
>>>> I'll file a separate bug about making scm.py's Git object use
>>>> --no-rebase during dcommit.
>>>> -eric
>>>>
>>>> On Thu, Nov 18, 2010 at 4:08 PM, Evan Martin <evan at chromium.org> wrote:
>>>>>
>>>>> On Thu, Nov 18, 2010 at 3:36 PM, David Levin <levin at google.com> wrote:
>>>>> >> It's some magical setup by which your git svn fetchs will be much
>>>>> >> faser.
>>>>> >>  But I've heard it's buggy?  Can lead to local repository
>>>>> >> corruption?
>>>>> >> Can someone set me straight?
>>>>>
>>>>> No magic, just standard git: complicated, but logical once you
>>>>> understand how it works. :\
>>>>>
>>>>> It means that both "git pull" and "git svn fetch" will be updating the
>>>>> same branch.  When the latter sees the former has pulled down new
>>>>> stuff (quickly, via the fast git protocol), it knows to rebuild its
>>>>> metadata from the new stuff you fetched (rather than fetching it all
>>>>> over again via the slow svn protocol).
>>>>>
>>>>> There's a catch: if you "git svn fetch", that creates new commits
>>>>> locally.  When you later "git pull", you get the commits that were
>>>>> constructed by git.webkit.org, which don't match (due to differing
>>>>> timestamps).  This may screw up rebase, but I believe it's smart
>>>>> enough to recognize the commits are "the same" (applied the same diff;
>>>>> in git jargon, they have the same patch-id).  In practice you don't
>>>>> even run "git svn fetch" except when the git server is behind, so I
>>>>> don't know if there are corner cases here that I haven't run into.  (I
>>>>> also haven't tried this on Windows in a while -- kind of terrified of
>>>>> git there, though I hear it works for others.)
>>>>>
>>>>> In particular for bots that are not committing, I see no catch, other
>>>>> than that they will be behind whenever git.webkit.org is behind.
>>>>>
>>>>> >> The current git svn fetch is *super* slow.  Especially if you're
>>>>> >> behind by
>>>>> >> more than a day or two.
>>>>> >> If there was a way to make this faster method safe, by wrapping it
>>>>> >> in some
>>>>> >> other (error-checking) command which knew how to fall back to git
>>>>> >> svn
>>>>> >> rebase, etc. when necessary I would love to make it the default
>>>>> >> method for
>>>>> >> all WebKit get users.
>>>>>
>>>>> I have instructed all Chrome git users to use this method since
>>>>> (checking the commit history...) Feb 2009 and it seems to work for us.
>>>>>  Note that you need git >= 1.6.1 or so for it to work properly.  I
>>>>> also use this method for working on WebKit (though I've only committed
>>>>> around 60 patches so I mostly use it for pulling down new code).
>>>>>
>>>>> PS: our tools also run svn dcommit with "--no-rebase" to avoid
>>>>> needlessly going out to svn again after committing.
>>>>
>>>>
>>>> _______________________________________________
>>>> webkit-dev mailing list
>>>> webkit-dev at lists.webkit.org
>>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>>>
>>>
>>
>
>