Page 1 of 1

Git vs Subversion Speed

Posted: Wed Feb 22, 2012 5:33 am
by ldo
Interesting to compare relative speeds:

* Pulling down the entire history of trunk/blender from (a total of about 24000 commits when I first did it) took about 5 hours using git-svn, as I recall. Pushing the resulting Git repo to my GitHub area took another 3 hours. Keeping that up-to-date only takes a few seconds each day.
* I am currently pulling down all the entries in tags, using the standard svn client. That’s a total of about 50 copies of the source tree. It’s been running for 22 hours, and done about half of them so far.

As another comparison, the entire size of my Git repo is currently 250MB. The checkout of the tags has getting close to 2GB, so it’ll probably be about 4GB when it’s finished, presumably sometime tomorrow.

To reinforce the point: a single checkout, equivalent to 50 commits, done with svn, is taking about an order of magnitude longer than a complete history dump of 24000 commits done via git-svn. Even if you try to argue that most of those commits only affected a small number of source lines, to the point where, say, those 50 code snapshots are really equivalent to 50000 commits, there still seems to be something of a disparity in speed of data transfer, don’t you think?

Posted: Wed Feb 22, 2012 5:38 am
by stiv
took about 5 hours using git-svn
Show us the numbers using a plan svn client.

Posted: Wed Feb 22, 2012 8:39 am
by ldo
The complete uncompressed Blender trunk source tree is currently about 90MB. So the tags checkout is trying to download about 50 separate copies of this. Which just points out how inefficient the implementation of tags is in Subversion.

Subversionistas like to brag about “cheap copies”, where it’s very quick and easy to create a new branch with minimal overhead on the server. But that’s not much help when you try to copy that entire new branch onto the client. And each tag in Subversion involves just such a copy.

By contrast, Git tags are nothing but pointers to commits, not copies of the commits. So once you already have a copy of the commit, copying the tag across the network is very quick and cheap.

Git tags can also be signed. This is handy for marking “official” releases. And it doesn’t matter where you got the objects in your repo from, you know they’re genuine if the signature verifies.

Posted: Wed Feb 22, 2012 7:17 pm
by stiv
So the tags checkout is trying to download about 50 separate copies of this.
Normally, an SVN user will simply check out trunk. I suspect what you are doing is downloading all the branches and tags. The branches are the expensive part.

Worse case, you are also getting the pre-built libraries for the various platforms.

Posted: Sat Feb 25, 2012 3:17 am
by ldo
It finally finished. There were in fact 60 different subdirectories in tags, totalling nearly 84GB, and they took about 87½ hours to download (just shy of 4 days).