Understanding Git - Part 1

Introduction

This is not a Git tutorial, I am not going to teach you how to set up a repository, add some files and push and pull. There are plenty of other very good tutorials out there which do that already.

This is about understanding Git - taking a step back for a bit, seeing what makes it tick and maybe getting a little insight into those mouth foaming fanatics who rave about it. Apologies in advance if you join them.

I've used many source control systems over the years: CMVC, CVS, Perforce, SourceSafe, Alienbrain and most recently Subversion. I've fallen into a pattern where I switch to a new project, find out what source control system they use and then look up a cheat sheet showing:

• how to get the latest code
• how do I send a changeset back to the server
• how do I revert a changeset

That done I get back to work having ideally spent as little time as possible understanding YASCS (Yet Another Source Control System).

Git, true to name, hasn't quite worked out like that for me. I've found my usual process falling down and I've had to invest a lot of time in understanding what's going on. Fortunately it's been worth it.

Hopefully this series of blog posts will save you some of the same effort.

First a little bit about hashes

I’m about to go into a lot of detail regarding hashes, it's pretty central to the whole concept of Git. Bear with me, or, if you already know all about hashes, skip ahead a bit.

The heart of Git is the concept of a Secure Hash Algorithm. A hash algorithm is simply method of generating a signature (referred to as a hash) representing the contents of a file.

Imagine you have an animated gif, but you're worried it might offend me so you don't want to send it over. We're chatting and you're pretty sure I've got the same one but you're not certain. Being fellow geeks we decide to use a hash. We both calculate the hashes of our files and then compare the numbers over chat. If the numbers don't match then we definitely don't have the same gif. Disaster averted. On the other hand, if they do match then it's highly likely we have the same file and you can safely send it over.

The quality of a hashing algorithm is determined by the number of unique hashes it can produce and its ability to scatter its results across that range. When two differing files produce the same hash, it's called a collision.

Git uses the 160-bit SHA-1 algorithm. The 160-bit number can be represented as a 40 digit hexadecimal number, such as this:

b9eb8044b706ab4f5b9ce6c5eeb7a406755eeabf

That's 1,461,501,637,330,902,918,203,684,832,716,283,019,655,932,542,976 possible signatures or 1 quindecillion 461 quattuordecillion 501 tredecillion 637 duodecillion 330 undecillion 902 decillion 918 nonillion 203 octillion 684 septillion 832 sextillion 716 quintillion 283 quadrillion 19 trillion 655 billion 932 million 542 thousand 976 if you want to read that out to a patient friend.

Linus Torvalds (the creator of Git and Linux) said the following about the chances of a hash collision when asked:

First off, let me remind people that the inadvertent kind of collision is really really really damn unlikely, so we'll quite likely never ever see it in the full history of the universe.

So that’s probably ok then. If you’re still worried, read the rest of his reply.

So what do hashes have to do with Git?

Unlike most version control systems, Git only cares about the contents of files. Two files can have totally different names and even locations, but if the data is the same, they’ll only be stored once. Git calls these ’blobs’.

You can see this if you look inside the .git/objects folder at the root of one of your repositories. In there you’ll see a list of folders with names in the hex range 00-FF. Git stores blobs in a subfolder here based on the first two characters of the name. Our example earlier would have the path .git/objects/b9/eb8044b706ab4f5b9ce6c5eeb7a406755eeabf.

This is a really important concept - you could have, say, 10,000 copies of the same file in a project. There will only be one copy stored in the .git folder to represent them as they all have the same hash.

Lastly comparing files by their hashes is incredibly fast - you can even do it via your eyeballs.

So what about the filenames and other data?

Each folder in Git is represented by a ‘tree’ file. A tree file is simply a plain text file representing a folder’s content with one row per entry. The rows are sorted and contain three pieces of information:

1. the name of the file or folder
2. the permissions
3. the hash of its blob file

You may be wondering at this point what the blob file of a folder would look like, it itself is another tree file stored in the .git/objects store just like any other blob file. Thus folder trees are recursive tree files. Any change to a file or folder in a subtree will cause its tree object to change, thus changing its parents tree file and so on, propagating all the way up to the root tree.

There are a number of important benefits to this:

• you can compare two huge trees of files just by comparing the hashes of their root tree objects
• you can move a complex subfolder from one location to another and only the parent tree objects will change, the data will not be re-stored

Commits

This leads us nicely to the third type of object in Git: commits. A commit, again, is simply another plain text blob file. A commit represents a snapshot in the system and contains (amongst a few other things):

• the hash of one or more parent commits (unless it’s the very first or root commit)
• the hash of the root tree object
• the details of the commit author
• the date of the commit (although this is fragile and can’t be trusted)
• a log message describing the commit

Tags

The final object type are tags. These are really just a way of giving a human readable alias to an existing object. Although they’re mostly used to name commits, they can be assigned to anything. Tags also have the ability to be cryptographically signed, certifying that the snapshot represented by that commit matches the one intended by the author.

Git add, staging files and the index

When working in a repository, there is something called an index. You can think of this as starting off as a copy of the tree file represented in the most recent commit. As you make changes in your working files this index... remains the same. You need to modify this index in preparation for your next commit. Changes added to the index are known as ‘staged’.

Running the command git status will give you a summary of two things:

1. the differences between your current index and the last commit
2. the differences between your working copy and the current index

This is quite a stumbling block at first (or at least it was for me). I’m used to working in version control systems where the status of my working copy is just one thing:

1. the differences between my working copy and the last commit

Having this staging ‘buffer’ in the middle is a very powerful feature. It allows you to only submit some of your changes to the repository whilst holding others back.

Let’s back up a little bit and see how this works. We know now that the contents of a file are stored in a blob and the location of that blob is determined by its hash. As we work on our files we make a modification to one of them. The hash of that file has how changed and the index no longer points to the correct blob.

At that point we need to run the command git add changedfilename to get it to calculate the new hash and add the blob to the object store. It will also modify the staging index to reference that blob.

Let’s say we don’t commit at that point and make some further changes to the file. If we then decide to commit we run into a slight problem - the blob we staged earlier no longer matches the one in the working copy as the hash has changed again. Thus we need to run git add changedfilename again before committing to update the index and object store with the new version.

So before committing, what we're really doing is updating this staging index and making sure it points to all the correct versions of the files you wish to check in. git add/rm/mv and other commands are all operating on updating and preparing this index.

Git diff and --cached

We now have this concept of three versions of your repository:

1. The last commit (HEAD)
2. The staged changes (the index)
3. Your working copy (the files you see on the disk)

Before splitting it down like this, doing diffs in Git can be quite confusing. You run git diff and only see some of your changes. What’s going on?

Git diff by default only shows you the changes between your unstaged files and those in the last commit. Any staged changes will be skipped from this view.

To view the diff between what you have staged and the last commit, instead run:

`git diff --cached`

A very useful command just before a commit to see what you're actually doing to the repository.

Conclusion

I've hopefully covered the foundation concepts of Git as far as a linear line of development goes. In parts of this series I’ll be looking at what it truly means for repositories to be distributed as well as (hopefully) demystifying what goes on in branches and merges.