Git: Distributed Version Control for Code and Documents

Git supports several protocols for accessing a remote repository, the most common three being Git protocol, SSH, and HTTP(S). Designed specifically for Git, the Git protocol favors data transfer by always transferring the smallest possible amount of data. It doesn’t support authentication, so it’s often transmitted over an SSH connection. This ensures both efficient (Git protocol) and secure (SSH) transmission. HTTP(S) is used when a firewall is configured very restrictively and the allowed ports are drastically restricted.⁠^[67]

In general, a valid URL contains the transfer protocol, the address of the server and the path to the repository:⁠^[68]

For the SSH protocol the short form still exists:

It is also possible to clone repositories locally using the following syntax:

If you want to know what URLs are configured for a remote repository, use git remote’s --verbose or -v option:

You can see that there are two URLs for the remote repository origin, but they are set to the same value by default. The first URL (fetch) specifies from where and with which protocol changes are downloaded. The second URL (push) specifies where changes are uploaded to and with which protocol. Different URLs are particularly interesting if you download or upload with different protocols. A common example is to download with the git protocol (git://) and upload with the SSH protocol (ssh://). It is then downloaded without authentication and encryption, which provides a speed advantage, but uploaded with authentication and encryption, which ensures that only you or other authorized people can upload. You can use the git remote set-url command to customize the URLs:

The current status of the remote repository is stored locally. Git uses the mechanism of remote tracking branches, special branches — local references — that reflect the state of the remote branches. They “track” the remote branches and are advanced or set by Git when synchronizing with the remote, if the branches in the remote have changed. In terms of the commit graph, remote tracking branches are markers within the graph that point to the same commits as the branches in the remote repository. You can’t modify remote tracking branches like normal branches; Git manages them automatically, so it updates them. When you clone a repository, Git initializes a remote tracking branch for each remote branch.

Figure 31, “Generated Remote Tracking Branches” shows an example. The origin remote repository has three branches: pu, maint, and master. Git creates a remote tracking branch in the cloned repository for each of these remote branches. It also creates a local branch master in the clone that corresponds to the remote branch master. This is checked out and is the branch you should work in if you plan to upload commits to the master (but see Sec. 5.3.1, “git fetch”).

In the git fetch example, there is only one branch on the remote side, master. That’s why Git creates only one remote tracking branch in the clone, origin/master. The git branch -r command shows all remote tracking branches:

The special entry origin/HEAD → origin/master states that in the remote repository the HEAD points to the branch master. This is important for cloning, because this branch is checked out after cloning. The list of remote tracking branches is a bit sparse in this example, you can see more entries in a clone of the Git-via-Git repository:

All branches can be displayed with git branch -a:

In this case, Git uses the prefix remotes/ to clearly distinguish remote tracking branches from normal ones. If you have enabled color output, the different branches will also be color-coded: the checked-out branch green, remote tracking branches red.

Remote Tracking Branches are also references only and are therefore stored under .git/refs like all references. However, since they are special references that are also linked to a remote repository, they end up under .git/refs/remotes/<remote-name> (see Sec. 3.1.1, “HEAD and Other Symbolic References”). In Gitk, the remote tracking branches are displayed with the prefix remotes/<remote-name>/, which is also colored dark yellow (Figure 32, “Branch next and the corresponding remote tracking branch in Gitk”).

As soon as new commits are created by other developers in a remote, you want to download them to your local repository. In the simplest case, you just want to find out which commits you don’t have locally, download them, and update the remote tracking branches so that they reflect the current status in the remote.

Use the git fetch command to do this:

Git acknowledges the call with a message that origin/master has been set from commit 79170e8 to commit 003e3c7. The notation master → origin/master indicates that the branch master from the remote was used to update the remote tracking branch origin/master. In other words: Branches from the remote on the left and remote tracking branches on the right.

See Figure 33, “Remote Tracking Branches are updated” for the effect this has on the commit graph: On the left side is the initial state of the remote origin and next to it that of the clone. Both the remote and the clone have new commits since the last synchronization (C and D). The remote tracking branch origin/master in the clone points to commit B; this is the last state of the remote known to the clone. By calling git fetch origin, Git updates the remote tracking branch in the clone to reflect the current status of the master (pointing to commit C) in the remote. To do this, Git downloads the missing commit C and then sets the remote tracking branch on it.

Suppose you want to transfer commits from the remote repository to your local branch. To do this, first run a git fetch to fetch new commits, and then merge the change from the corresponding remote tracking branch:⁠^[71]

For this use case, Git provides the git pull command to speed up your workflow. It is a combination of git fetch and git merge or git rebase.

Downloading new commits from origin and merging all commits referenced by the master there into the current branch can be done with the following command:

In Figure 34, “What happens with a pull” we illustrate the process. On the left, you see the remote repository origin and next to it the current status of the local repository. The repository was cloned when it only contained commits A and B, so the remote tracking branch points origin/master to B. In the meantime, both the remote (C) and local (D) repositories have been added.

On the right side is the state after git pull origin master. Commit C has been added to the local repository. The fetch call contained in the pull has updated the remote tracking branch, i.e. it points to the same commit as the master in origin and thus reflects the state there. In addition, the merge call contained in the pull has integrated the master from origin into the local master, as you can see from the merge commit M and the current position of the local master.

Alternatively, the --rebase option instructs the pull command to rebase the local branch to the remote tracking branch after fetch:

In Figure 35, “What happens during a pull with rebase” you can see what happens if you perform a rebase instead of the default merge.

The initial situation is the same as in Figure 34, “What happens with a pull”. The fetch contained in the pull moves the remote tracking branch origin/master to commit C. However, rebase does not create a merge commit; instead, a call to rebase gives the commit D a new base, and the local master is set to the new commit D'. (Rebase is described in detail in Sec. 4.1, “Moving commits — Rebase”).

Git beginners often ask themselves whether they should use fetch or pull. The answer depends on how you develop: How big is the project? How many remotes are there? How heavily are branches used?

There are two ways to delete references in the remote: The older one (before Git version 1.7.0) is to omit the local reference in the refspec — this statement means you want to upload “nothing”. So you replace an existing reference with the empty one.

However, newer git versions usually use the git push command with the --delete option, which is syntactically much clearer:

Note that in other clones, the remote tracking branch origin/bugfix, if present, does not automatically disappear! See the section on pruning above (Sec. 5.3, “Downloading Commits”).

In everyday life you often run git push without specifying remote and refspec. In this case, Git uses the configuration entries (upstream branch and push.default) to decide which references are sent where.

By default, Git proceeds like this:⁠^[73] If you don’t specify a remote, Git will look for the upstream configuration of the current branch. If the name of the branch on the remote side matches the name of the local branch, the corresponding reference is uploaded (this is to protect you from uploading, for example, your branch devel to master if the upstream configuration is incorrect). If no upstream branch is configured, Git aborts with an error message:

If you use git push <remote> to specify a remote but no branch, Git will attempt to upload the current branch to the remote under the same name.

The strategy described here is also known as simple. For most use cases, it does what the user expects and protects against avoidable errors. However, you can set the push.default option responsible for this to one of the following values if required:

In some cases, Git will automatically configure upstream branches (for example, after a git clone). However, you need to do this explicitly, especially for new branches that you are uploading for the first time. You can do this either afterwards using the --set-upstream-to option or, in short, -u of git branch:

Alternatively, and if you think about it, you can also have git push write the configuration when you call git push with the -u option:

To view the upstream configuration of your branches, call git branch -vv. The output shows the upstream partner of a branch (if any) in square brackets.

The git remote show command gives a concise summary of the remote, including the branches available there, whether they are tracked locally (tracking status) and which local branches are configured for specific tasks.

The command must request the current status from the remote, i.e. the command fails if the remote is not available, e.g. due to a missing network connection. The option -n prevents the query.

If you have configured an upstream branch, when you change the branch (git checkout) and query the status (git status), you will receive a notification about the status of the branch compared to the upstream, for example:

Here there are four different possibilities:

With the -v (compare only) or -vv (compare and upstream name) option, git branch displays the appropriate information for local branches:

The command prints the SHA-1 prefix for all branches and the commit message of the current commit. If an upstream is configured for the branch, Git returns both the name and a comparison to the upstream. In the example, you see four different branches. master has an additional commit that has not yet been uploaded to the remote, and is therefore ahead. The branch feature, on the other hand, has no upstream branch configured, so it currently exists only locally. The branch next is up-to-date with the corresponding remote tracking branch. The Branch pu, on the other hand, “lags” behind its upstream and is therefore displayed as behind. The only state missing here is diverged — then both ahead and behind are shown including the number of “missing” commits.

To generate a pull request automatically, there is the git command request-pull. The syntax is:

As <URL> you specify your public repository (either as the actual URL or as a configured remote repository), and as <start> you select the reference on which the feature is built (in many cases the branch master, which should match the master branch of the official repository). Optionally, you can specify an <end>; if you omit this, Git will use HEAD.

The output is by default STDOUT, and includes the repository’s URL and branch name, a short description of all commits by author, and a diff state, i.e., a balance of added and deleted lines by file. This output can easily be forwarded to an e-mail program. If you add the -p option, a patch with all changes is appended below the text.

For example, to ask someone to download the two latest commits from a repository:

In principle, Git automatically downloads new tags when you call git fetch or git pull. That is, if you download a commit that has a tag pointing to it, that tag will be included. However, if you use a refspec to exclude individual branches, then commits in those branches will not be downloaded, and thus no tags that may point to those commits will be downloaded. Conclusion: Git only downloads relevant tags. With the options --no-tags (no tags) and --tags or -t (all tags) you can adjust the default behavior. Note, however, that --tags not only downloads the tags, but necessarily the commits to which they point.

Git notifies you when new tags arrive:

If you want to know what tags are present on the remote side, use git ls-remote with the --tags option. For example, you can get all release candidates of git version 1.7.1 with the following call:

Git outputs the SHA-1 sums of the tags and their contents.⁠^[75]

Git does not automatically upload tags. You need to pass them explicitly to git push, similar to the branches, e.g. to upload the tag v0.1:

If you want to upload all tags at once, use the --tags option. But be careful: Avoid this option if you use Annotated Tags to mark versions and Lightweight Tags to mark something locally, as described in Sec. 3.1.3, “Tags — Marking Important Versions”, because with this option you would upload all tags, as already mentioned.

Attention: Once you have uploaded a tag, you should never change it! The reason: Let’s say Axel changes a tag, like v0.7, that he has already released. First it pointed to the 5b6eef commit, and now to bab18e. Beatrice had already downloaded the first version pointing to 5b6eef, but Carlos had not yet. The next time Beatrice calls git pull, Git won’t download the new version from the v0.7 tag; the assumption is that tags don’t change, so Git doesn’t check the validity of the tag! When Carlos now runs git pull, he also gets the v0.7 tag, but it now points to bab18e. Finally, two versions of the tag — each pointing to different commits --- are in circulation. Not a very helpful situation. It gets really confusing when both Carlos and Beatrice use the same public repository, and upload all tags by default.⁠^[76] The tag “jumps” back and forth between two commits in the public repository, so to speak; which version you get with a clone depends on who pushed last.

If you do get this mishap, you have two options:

The git format-patch command exports one or more commits as patches in Unix mailbox format and prints one file per commit. The file names consist of a sequential numbering and the commit message, and end in .patch.⁠^[77] As an argument, the command expects either a single commit or a range such as A..B. If you specify a single commit, Git will evaluate this as the selection from the commit to the HEAD.

Figure 38, “Formatting three commits to 'master' as patches” shows the initial situation. We want to export the three commits in the fix-git-svn-docs branch, that is, all commits from master, as patches:

The generated files contain, among other things, the header fields From, Date and Subject, which are used for sending as e-mail. These fields are completed using the information available in the commit — author, date, and commit message. The files also contain a diff-stat summary and the changes themselves as a patch in unified diff format. The [PATCH m/n] suffix⁠^[78] in the subject line is used later by Git to apply the patches in the correct order.

A corresponding excerpt follows:

If you plan to send a series of patches, it is recommended that you use the --cover-letter option to create a kind of “cover page” in which you describe the series. By default the file is called 0000-cover-letter.patch. Apart from the default headers, such a file looks like this:

As you can see, the Subject: still has the prefix [PATCH 0/3]; this way, all recipients can immediately see that it is a cover page. The file also contains the output of git shortlog and git diff --stat. Replace * SUBJECT HERE with a subject and BLURB HERE * with a summary of the patch series. Send the file together with the patch files.

Send the generated files with git send-email (or an email client of your choice). The command expects as its only mandatory argument either one or more patch files, a directory full of patches, or a selection of commits (in which case Git also calls git format-patch internally):

The command git send-email sets the fields Message-Id and In-Reply-To. This makes all e-mails after the first one look like replies to them and thus most mail programs will display them as a continuous thread:⁠^[79]

You can customize the command with options such as --to, --from and` --cc` (see the git-send-email(1) man page). However, if not specified, the essential information is queried interactively — most important is an address to which the patches should be sent.⁠^[80]

Before the emails are actually sent, you will see the header again; you should check if everything is as you want it, and then answer the question` Send this email? ([y]es|[n]o|[q]uit|[a]ll):` answer with y for “yes”. To get familiar with the command, you can first send all emails only to yourself or use the --dry-run option.

If you want to use your preferred Mail User Agent (MUA) (e.g. Thunderbird, Kmail or others) to send patches, there may be a few things to consider. Some MUAs are notorious for mutilating patches so that Git won’t recognize them as such.⁠^[83]

Patch emails exported with git format-patch are translated back into commits by the git command git am (apply from mailbox). A new commit is created from each email, and its meta-information (author, commit message, etc.) is generated from the email header lines (From, Date). As mentioned earlier, Git uses the number in the subject to determine the order in which the commits should be entered. To complete the example from earlier: If the emails are in the Maildir directory patches, then that’s enough:

When you apply patches from others using git am, the values of Author/AuthorDate and Committer/CommitDate are different. This means that both the author of the commit and the person who commits it are honored. In particular, the attributes are retained; it remains traceable who wrote which lines of code. With Gitk, the author and committer values are displayed by default; on the command line, set the --format=fuller option, which is accepted by git log and git show, among others:

With the Dictator and Lieutenants Workflow (Sec. 5.10, “A Distributed, Hierarchical Workflow”), it can happen that more than two people are involved in a commit. In this case, it makes sense that everyone who reviews the patch also “approves” it, especially the author. For this purpose, there is a --signoff option (-s for short) for the git commit and git am commands, which appends the committer’s name and email to the commit message:

This feature is especially useful for larger projects, which usually have guidelines on how to format commits and how best to send them.⁠^[84]

Conflicts can occur when patches are entered with git am, e.g. if the patches are based on an older version and the lines concerned have already been changed. In this case, the process is interrupted and you then have several options for how to proceed. Either resolve the conflict, update the index and continue the process with git am --continue, or skip the patch with git am --skip. Use git am --abort to abort the process and restore the current status of the branch.

Because patches usually contain changes made by others, it can sometimes be difficult to find the right solution to a conflict. The best strategy for patches that cannot be applied is to ask the author of the patches to rebase them to a well-defined base, such as the current master, and send them again.

Submodules are managed by Git as subdirectories that have a special entry in the .gitmodules file. The command git submodule is responsible for handling them.

First we need to import the library. This is done with the following command:

From the output of git status we can now see that there is a new directory libgit2 and that the file .gitmodules with the following content has been created

This file has already been added to the index, prepared for committing. The libgit2 directory, on the other hand, does not appear in the output of git diff --staged as usual:

Instead of listing all the files in the directory, Git saves a “special” file (recognizable by the unusual file mode 160000) that simply records the commit the module is currently on.

We import these changes, and from now on we can compile libgit2 in its subdirectory and then link against it:

The parent project and libgit2 are now merged in the working tree, but their version history is and remains separate. In the Git repository of libgit2 you can behave exactly the same way as in a “real” repository. For example, you can look at the output of git log in the parent project and after a cd libgit2 in the submodule.

Unlike submodules, which maintain their character as a standalone Git repository, when you work with Subtrees, you directly merge the history of two projects. A comparison of the two approaches follows.

Essentially, this technique is based on so-called subtree-merges, which were briefly discussed in Sec. 3.3.3, “Merge Strategies” about merge strategies. In our example, a subtree-merge is done by merging regular commits from the libgit2 repository under the libgit2/ tree (directory) — a top-level file in the library repository thus becomes a top-level file in the libgit2/ tree, which in turn is part of the repository.

Git has a command to manage subtree-merges.⁠^[86] You must always explicitly specify which subdirectory you are referring to by using -P <prefix>. To import the libgit2 in version 0.8.0, use:

The command automatically downloads all required commits and creates a merge commit that creates all the files of libgit2 under the directory libgit2/. The merge commit now links the previous version history to that of libgit2 (by referencing an original commit and then referencing other commits).

The result of this procedure is that your repository now contains all relevant commits from libgit2. Your repository now has two root commits (see also multi-root repositories in Sec. 4.7, “Multiple Root Commits”).

The files are now stored inseparably linked to the project. A git clone of this repository would also transfer all files under libgit2.⁠^[87]

Now what happens when you want to “upgrade” to v0.10.0? Use the pull command from git subtree for this:

Note: Since the original libgit2 commits are present, these commits also seem to change top-level files (e.g., COPYING when you use git log --name-status to examine the version history). In fact, these changes are actually made in libgit2, which is the responsibility of the merge commit, which aligns the trees accordingly.

The question “Submodules or Subtrees?” cannot be answered in general, but only on a case by case basis. The decisive criterion should be the affiliation of the subproject to the superordinate one: If you include third-party software, it is probably more likely to be submodules, your own with limited commits and a direct relationship to the main project rather than a subtree.

For example, when you install CGit (see Sec. 7.5, “CGit — CGI for Git”), you must initialize and update a submodule to compile libgit.a. So CGit needs the source code of Git, but doesn’t want to merge the development history with that of Git (the comparatively few CGit commits would be lost in this!). You can, however, compile CGit against another version of Git if you wish — the flexibility of the sub-repository is preserved.

The graphical repository browser Gitk, on the other hand, is managed as a subtree. It is developed in git://ozlabs.org/~paulus/gitk, but is included in the main Git repository with the subtree-merge strategy under gitk-git/.