[webkit-dev] Faster Git SVN updates

Eric Seidel eric at webkit.org
Tue Nov 30 14:20:44 PST 2010

I've now posted a patch to fix update-webkit as well:

Once that lands, I'll move the EWS bots over to using this "new" setup.
 Assuming those stay working, we can teach the tools to offer to fix "old"


On Mon, Nov 22, 2010 at 4:58 PM, Ojan Vafai <ojan at chromium.org> wrote:

> http://trac.webkit.org/changeset/72575 for having scm.py do the right
> thing.
> On Thu, Nov 18, 2010 at 4:11 PM, Eric Seidel <eric at webkit.org> wrote:
>> OK.  Sounds we should make this setup default.  I'll see if we can't
>> educate update-webkit and webkitpy/common/checkout/scm.py about detecting
>> this setup and doing the right thing in that case.
>> I'll file a separate bug about making scm.py's Git object use --no-rebase
>> during dcommit.
>> -eric
>> On Thu, Nov 18, 2010 at 4:08 PM, Evan Martin <evan at chromium.org> wrote:
>>> On Thu, Nov 18, 2010 at 3:36 PM, David Levin <levin at google.com> wrote:
>>> >> It's some magical setup by which your git svn fetchs will be much
>>> faser.
>>> >>  But I've heard it's buggy?  Can lead to local repository corruption?
>>> >> Can someone set me straight?
>>> No magic, just standard git: complicated, but logical once you
>>> understand how it works. :\
>>> It means that both "git pull" and "git svn fetch" will be updating the
>>> same branch.  When the latter sees the former has pulled down new
>>> stuff (quickly, via the fast git protocol), it knows to rebuild its
>>> metadata from the new stuff you fetched (rather than fetching it all
>>> over again via the slow svn protocol).
>>> There's a catch: if you "git svn fetch", that creates new commits
>>> locally.  When you later "git pull", you get the commits that were
>>> constructed by git.webkit.org, which don't match (due to differing
>>> timestamps).  This may screw up rebase, but I believe it's smart
>>> enough to recognize the commits are "the same" (applied the same diff;
>>> in git jargon, they have the same patch-id).  In practice you don't
>>> even run "git svn fetch" except when the git server is behind, so I
>>> don't know if there are corner cases here that I haven't run into.  (I
>>> also haven't tried this on Windows in a while -- kind of terrified of
>>> git there, though I hear it works for others.)
>>> In particular for bots that are not committing, I see no catch, other
>>> than that they will be behind whenever git.webkit.org is behind.
>>> >> The current git svn fetch is *super* slow.  Especially if you're
>>> behind by
>>> >> more than a day or two.
>>> >> If there was a way to make this faster method safe, by wrapping it in
>>> some
>>> >> other (error-checking) command which knew how to fall back to git svn
>>> >> rebase, etc. when necessary I would love to make it the default method
>>> for
>>> >> all WebKit get users.
>>> I have instructed all Chrome git users to use this method since
>>> (checking the commit history...) Feb 2009 and it seems to work for us.
>>>  Note that you need git >= 1.6.1 or so for it to work properly.  I
>>> also use this method for working on WebKit (though I've only committed
>>> around 60 patches so I mostly use it for pulling down new code).
>>> PS: our tools also run svn dcommit with "--no-rebase" to avoid
>>> needlessly going out to svn again after committing.
>> _______________________________________________
>> webkit-dev mailing list
>> webkit-dev at lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20101130/6634f244/attachment.html>

More information about the webkit-dev mailing list