Previous Thread  Next Thread

chat icon Git vs Subversion Speed

ldo

Posted: Wed Feb 22, 2012 5:33 am
Joined: 07 Nov 2010
Posts: 544
Interesting to compare relative speeds:

* Pulling down the entire history of trunk/blender from svn.blender.org (a total of about 24000 commits when I first did it) took about 5 hours using git-svn, as I recall. Pushing the resulting Git repo to my GitHub area took another 3 hours. Keeping that up-to-date only takes a few seconds each day.
* I am currently pulling down all the entries in tags, using the standard svn client. Thatís a total of about 50 copies of the source tree. Itís been running for 22 hours, and done about half of them so far.

As another comparison, the entire size of my Git repo is currently 250MB. The checkout of the tags has getting close to 2GB, so itíll probably be about 4GB when itís finished, presumably sometime tomorrow.

To reinforce the point: a single checkout, equivalent to 50 commits, done with svn, is taking about an order of magnitude longer than a complete history dump of 24000 commits done via git-svn. Even if you try to argue that most of those commits only affected a small number of source lines, to the point where, say, those 50 code snapshots are really equivalent to 50000 commits, there still seems to be something of a disparity in speed of data transfer, donít you think?
Reply with quote


stiv

Posted: Wed Feb 22, 2012 5:38 am
Joined: 05 Aug 2003
Posts: 3645
Quote:
took about 5 hours using git-svn


Show us the numbers using a plan svn client.
Reply with quote


ldo

Posted: Wed Feb 22, 2012 8:39 am
Joined: 07 Nov 2010
Posts: 544
The complete uncompressed Blender trunk source tree is currently about 90MB. So the tags checkout is trying to download about 50 separate copies of this. Which just points out how inefficient the implementation of tags is in Subversion.

Subversionistas like to brag about ďcheap copiesĒ, where itís very quick and easy to create a new branch with minimal overhead on the server. But thatís not much help when you try to copy that entire new branch onto the client. And each tag in Subversion involves just such a copy.

By contrast, Git tags are nothing but pointers to commits, not copies of the commits. So once you already have a copy of the commit, copying the tag across the network is very quick and cheap.

Git tags can also be signed. This is handy for marking ďofficialĒ releases. And it doesnít matter where you got the objects in your repo from, you know theyíre genuine if the signature verifies.
Reply with quote


stiv

Posted: Wed Feb 22, 2012 7:17 pm
Joined: 05 Aug 2003
Posts: 3645
Quote:
So the tags checkout is trying to download about 50 separate copies of this.


Normally, an SVN user will simply check out trunk. I suspect what you are doing is downloading all the branches and tags. The branches are the expensive part.

Worse case, you are also getting the pre-built libraries for the various platforms.
Reply with quote


ldo

Posted: Sat Feb 25, 2012 3:17 am
Joined: 07 Nov 2010
Posts: 544
It finally finished. There were in fact 60 different subdirectories in tags, totalling nearly 84GB, and they took about 87Ĺ hours to download (just shy of 4 days).
Reply with quote


 
Jump to:  
Powered by phpBB © 2001, 2005 phpBB Group