Git: Distributed Version Control for Code and Documents

Git is designed for decentralized repository management; the smallest unit for exchanging changes between repositories is commits. However, since there are sometimes thousands of commits between two versions of a piece of software, and a single commit-by-commit transfer would generate a lot of overhead, commits are grouped together into so-called packfiles before being transferred. These packfiles are a simple but effective format.⁠^[92] They are also used to store (older) commits on the hard disk in a space-saving way (git gc or git repack, see Sec. B.1, “Cleaning Up”).

These packfiles are usually transmitted via the Git protocol, which runs on port 9418/TCP by default. The Git protocol is deliberately kept very simple in design and offers only a few functions that are directly related to the structure of Git: What data to send or receive, and a way for the sender and receiver sides to agree on the smallest possible amount of data that needs to be transmitted to synchronize both sides.

Therefore, the Git protocol does not include any authentication option. Instead, Git uses an already existing, secure and simple authentication structure: SSH, the 'Secure Shell'.

So while the Git protocol can be used unencrypted and in raw form for anonymous read-only access, writing or uploading via Git protocol only works via SSH.

Git also supports transport via HTTP(S), FTP(S), and Rsync. Although Rsync is now considered deprecated, and should not be used anymore, there are some use cases for HTTP(S): In particularly restrictive environments with very restrictive firewall rules, you may be able to access a repository via HTTP(S) (i.e. only on port 80 or 443) for both read and write operations. Platforms like GitHub (see Ch. 11, GitHub) therefore offer HTTPS as default transport method.

If you want to synchronize changes to repositories on the same computer, this does not have to be done via detours: Git communicates directly with the other side via Unix pipes, negotiates a common basis and synchronizes the data. (Of course, this requires that the user invoking the Git command has at least read permission to the other repository’s pack files).

So far, you’ve probably worked mostly with Git repositories, which were working tree and repository in one: The repository-internal data is stored in the .git subdirectory, all other files belong to the working tree, i.e. you can edit them while Git observes and stores the changes to these files (tracking).

A so-called bare repository, i.e. a “mere” repository, has no assigned working tree. It contains only the files and directories that are stored in a “regular” repository under .git.

You create such a bare repository with git init --bare. Take a look at the difference between the two options:

To create a backup of one of your normal repositories, you can create a new bare repository (e.g. on a USB stick) and upload all your references (and thus all your commits):

With git init the files are usually created with read and write permission according to the umask setting. This is also a convenient choice for the end user. However, if you want to set up a repository on a server, you can use the --shared option to specify who (at the filesystem level) can access the repository.

When you initialize a repository with --shared, the receive.denyNonFastForwards option is automatically set. It prevents uploading commits that cannot be integrated via Fast-Forwards (even if the user explicitly wants to via git push -f).

Usually, write access to Git repositories located on another computer can only be granted via SSH. However, it is generally undesirable to grant a user who will have access to a repository the same user rights to the whole system.

Git works around this problem with the included git-shell program. It works like a shell, but only allows you to run four Git commands that are responsible for uploading and downloading pack files. Interactive use or execution of other commands is denied by the shell unless you explicitly enable the “Interactive Mode” of the shell — see the git-shell(1) man page for details.

If you create a new user and assign the git shell to him, e.g., using chsh <user>, he cannot log in via SSH, but he can upload commits to all git repositories to which he has write permission.

It’s a major advantage that Git uses SSH as an encrypted and authenticated transport channel, because most users already have a key pair (public/private) with which they log in on other computers.

So instead of tediously assigning (and then sending out) passwords to accounts, a system administrator can limit access to Git repositories to users who authenticate against SSH public keys. This saves time for the user (by eliminating the need to re-enter a password), but also saves the administrator from having to worry about password changes (which would not be easily possible using the Git shell).

In the following we will show you how to set up two users on your system, max and moritz, and let them work on the same repository.

First, we have to set up a repository that the two users will want to access later. Assuming that other repositories might follow later, we will create a Unix group git (generally for Git users) and a directory /var/repositories with read permission for members of the git group, as well as a git-example group and its corresponding directory, writeable only for members of git-example, in which the repository will later be located:

We also create a repository in the last created directory:

Next we create the two users. Note that this call will not create a home directory for the users under /home/. Also, both are added to the git and git-example groups:

Next, we have to assign a password to each user via passwd so that they can log in via SSH. Afterwards, the new users can now work together on a project. You add the remote as follows:

All other users who want to contribute to this project must belong to the git-example group. So this approach is essentially based on the use of Unix groups and Unix users. However, a server admin usually wants to offer not only Git, but various services. And to control the user administration entirely via Unix groups is rather inflexible.

The installation of Gitolite is simple. All you need to do is have your public key ready to register as an administrator. You don’t need root privileges unless you need to create the git user first,⁠^[95] so skip the next step if you have already created such a user.

First, create a user on the computer that will act as the git server (henceforth <server>). Usually, this user is called git, but you may also call it something else (e.g. gitolite). You can specify /home/git as your home directory or, as in this example, something like /var/git:

Now switch to the git user. Gitolite needs the .ssh/ and bin/ directories, so we need to create them:

Now clone the Gitolite repository and install a symlink to bin (this is already the whole installation):

You can now configure Gitolite and enter your public key with which you want to manage the Gitolite configuration:

Check that Gitolite works on the computer where you normally work (and where you have stored the corresponding private key):

You should verify that your key gives you read and write permission to the gitolite-admin repository. Now clone it onto your computer:

The repository contains the entire configuration for Gitolite. You check in your changes there and upload them via git push: the server automatically updates the settings.

The Gitolite admin directory contains two subdirectories, conf and keydir. To introduce a new user to Gitolite, you need to put their SSH key under keydir/<user>.pub. If the user has multiple keys, you can store them in separate files of the format <user>@<description>.pub:

Don’t forget to check in the new keys with git add keydir followed by git commit. To make them known to the gitolite installation, you also need to upload the commits using git push.

Then you can assign permissions to this username in the conf/gitolite.conf configuration file.

You can save yourself a lot of administrative work and typing by using macros. You can combine groups (of users or repositories), e.g.:

These macros are also evaluated recursively. When defining them, it does not have to be clear whether they are users or repositories; the macros are only evaluated at runtime. This allows you to create groups from other groups:

There is a special group @all which, depending on the context, contains all users or all repositories.

You can configure one (or more) repositories as follows:

R and W stand for read or write access. The plus means that forced uploading is also allowed (non-fast-forward, i.e. also deleting commits).

For a repository, of course, several such lines can be entered. In a small project there could be maintainers, other developers and testers. Then the access rights could be regulated as follows:

Thus, the testers have read-only access, while the developers are allowed to upload new commits, but only if they can be integrated via fast-forward. The main maintainers are allowed “everything”.

These lines are processed sequentially. If the line applies to a user, Gitolite authorizes the user and grants him the appropriate rights. If no line matches the user, the user is rejected and is not allowed to change anything in the repository.

A user can view all his permissions by simply logging into the Git server via SSH. Immediately after installation, this is how it looks like to the administrator:

If you want to install a web-based tool to browse the Git repositories later, you should also name a person in charge and describe the project:

For this to work, you must first enable Gitolite to set these config settings: This is done on the server where Gitolite is installed, in the file .gitolite.rc: Enter the value gitweb\..* under the GIT_CONFIG_KEYS key.

Especially in corporate environments, access rights often have to be differentiated even more finely than a mere “has access” and “must not access”. For this purpose, Gitolite offers access restriction on directory- and file- as well as tag- and branch-level.

We will first look at a case that occurs frequently: developers should be able to develop on development branches at will, but only a small group of maintainers should be able to edit “important” branches such as master.

This could be implemented in a similar way:

Here a “development namespace” is created: The group of developers can work with branches below dev/, e.g. create dev/feature or delete it. However, the developers can only read the master branch, not change it — this is reserved for the maintainers.

The part between the flags (RW+) and the equal sign is a so-called Perl-Compatible Regular Expression (PCRE). If it does not start with refs/, the expression refers to all references below refs/heads/, i.e. branches. In the above example, any references below refs/heads/dev/ can be modified — but not the dev branch itself, nor anything-dev!

But if such an expression starts explicitly with refs/, you can manage any references. In the following way you can set up that all maintainers are allowed to create Release Candidate tags,⁠^[96] but only one maintainer is really allowed to create the versioning tag (or any other):

If one of the maintainers still wants to upload a tag like v1.0, the following happens:

As mentioned above, here the rules are applied one after the other. Since the tag v1.0 does not match the regular expression above, only the bottom line comes into question, but the username does not match. No line is left (fallthru), so the action is not allowed.

The concept of personal namespaces is somewhat more flexible. This gives each developer his own hierarchy of branches that he can manage.

There is a special keyword for this, USER, which is replaced by the user name currently accessing the branch. This makes the following possible:

Now all developers under p/<user>/ can manage their branches as they like. The lower directive makes sure that all developers can read these branches. Now max can e.g. create p/max/bugfixes, but moritz can only read them.

Gitolite also allows file- and directory-level access restrictions. The virtual reference VREF/NAME is responsible for this. For example, you can allow the documentation team only (writing⁠^[97]) access to doc/:

However, the following pitfalls must be taken into account: Once the keyword VREF/NAME appears once, the file-based rules are applied to all users. If none of them apply, access is allowed — so the second rule is important, which prohibits access for @doc unless the commit only modifies files under doc/ (see also Sec. 7.2.7, “Explicitly Prohibiting Actions” below).

Access control checks at the commit level which files are modified; if a commit contains changes to a file that the user is not allowed to edit, the entire push process is aborted. In particular, no actions can be performed that involve commits from other developers that modify files outside the allowed range.

Specifically, in relation to the above example, this means that @doc members generally cannot create new branches. Creating a new branch would mean creating a new reference to an initial commit and then fast-forwarding all commits from top to root, i.e., the entire project history. However, there are certainly commits in it that modify files outside of doc/, and so the action is prohibited.

Previously, a user was only rejected if he failed all rules (fallthru), i.e. if no rights were assigned to him. But the - flag (instead of RW) can be used to explicitly restrict access. Again, the rules are passed through from top to bottom.

This directive prohibits members of @developers from making commits that modify the Makefile.⁠^[98]

By convention, you should never upload forced updates to the master or maint branches (see also Sec. 3.1, “References: Branches and Tags”). You can now force this policy with Gitolite:

If a branch that is not called master or maint is uploaded, only the third rule is applied and arbitrary access (including non fast-forward updates) is allowed. Commits that can be integrated to master or maint via fast-forward are allowed by the first rule. Note the missing plus sign, though: A forced update is not covered by the first rule, but by the second one, which explicitly prohibits everything (that has not been allowed before).

With the means presented here and others, which you can take from the documentation, ⁠^[99] you are able to force policies very flexibly. However, it may not be useful to control everything down to the smallest detail. As mentioned above, especially a control on file name level is problematic. Then, if hours of work go into a commit, but it can’t be uploaded because one of those restrictions prohibits it, the frustration is great (and fixing that commit is not trivial, either; see rebase, Sec. 4.1, “Moving commits — Rebase”).

At the branch level, it makes sense to give only a limited group of developers access to “important” branches (such as master). Of course, strict control over who can do what comes at the expense of flexibility, and it’s this flexibility that makes branching in Git so practical.

As a rule, the Git daemon is supposed to constantly deliver a large number of repositories. To do this, it runs constantly in the background or is restarted for each request. The latter task is typically performed by Inetd from OpenBSD. To make this work, you just need to add the following (one!) line to /etc/inetd.conf:

<user> must be a user who has read access to the repositories. This can be root, because the Inetd normally runs with root privileges, but should be git or a similarly unprivileged account.

The configuration for the xinetd is similar, but more self-explanatory. It is stored e.g. under /etc/xinet.d/git-daemon:

Do not forget to restart the respective daemon via /etc/init.d/[x]inetd restart.⁠^[101]

Debian offers the git-daemon-run package which contains configuration files for sv.⁠^[102] The package essentially creates a gitlog user and two executable shell scripts, /etc/sv/git-daemon/run and /etc/sv/git-daemon/log/run. Modify the former to run the Git daemon in the directory where your repositories are located:

If you start the Git daemon from a shell script this way (or similarly via SysV-Init), the script will be executed with root privileges. The following options are therefore useful:

If you are using SysV-Init, which means that services are usually started via symlinks in /etc/rc2.d/ to scripts in /etc/init.d/, you will also need to create the following symlinks to automatically start the git daemon when the system boots

On a production system that is more than just a Git server, you may encounter the following situations:

The Git daemon provides options to respond to such situations. They are summarized below. For more detailed explanations, please consult the git-daemon man page.

Gitolite knows a special username, daemon. For all repositories where this user has read permission, the file git-daemon-export-ok is automatically created. So you can use Gitolite to directly specify which repositories to export:

Note that this setting has no effect if you start the Git daemon with the --export-all option. Also, you cannot give this permission to all repositories via repo @all.

Many distributions already include Gitweb as a separate package or directly in the Git package. Under Debian, the corresponding package is called gitweb. If you are not sure if Gitweb is available on your system, you should check under /usr/share/gitweb and install it if necessary.

Gitweb only requires a large Perl script plus a configuration file and optionally a logo, CSS stylesheet, and favicon. The configuration file is usually located in /etc/gitweb.conf, but can also be named differently. It is important that each time the Perl script is called, the environment variable GITWEB_CONFIG is used to specify where this file is located.

Usually you should already have such a file. The following list shows the most important configuration options.

Attention: The file must be written in valid Perl. In particular, do not forget the concluding semicolon when assigning variables!

If you installed Gitolite as mentioned above, and your repositories are located under /var/git/repositories, the following Gitweb configuration should be sufficient:

Assuming that you have installed the CGI script under /usr/lib/cgi-bin and the image and CSS files under /usr/share/gitweb (as the Debian gitweb package does), configure Apache as follows:

Create /etc/apache2/sites-available/git.example.com with the following content:

Then you need to activate the virtual host and let Apache reload the configuration:

Depending on how you implement virtual hosts in Lighttpd, the configuration might look different. Three things are important: That you make aliases for the globally installed Gitweb files, set the environment variable GITWEB_CONFIG and that CGI scripts are executed. To do this you need to load the modules mod_alias, mod_setenv and mod_cgi (if you haven’t already done so).

The configuration then looks like this:⁠^[104]

The integration in Apache and Lighttpd is similar. However, since CGit uses “nicer” URLs (like http://git.example.com/dwm/tree/dwm.c for the dwm.c file from the dwm repository), a little effort is required to rewrite the URLs.

The following configurations run CGit on git.example.com:

For Lighttpd you have to resort to some tricks. You must not forget to configure virtual-root=/ (see below — this setting is not harmful for Apache either).

The configuration is controlled by the file /etc/cgitrc. A list of supported options can be found in the file cgitrc.5.txt in the source directory of CGit (unfortunately the program does not include any other documentation). The most important ones are listed below:

With the following configuration, any repository you have allowed Gitweb access to in Gitolite will appear in the listing — and the description and author (if specified, see Sec. 7.2.3, “Ownership and Description”) will also be displayed:

With the scan-path option explained above, in combination with Gitolite it is usually not necessary to add and configure repositories individually. However, if you want to do this, or if your repositories are not stored in a central location, you can do this per repository as follows:

For more repository-specific configurations, consult the sample configuration file or the explanations of the options in the cgitrc.5.txt file in the source directory of CGit. You can also group these manually configured repositories under different sections (option section).

CGit is especially fast compared to Gitweb because it is written in C and also supports caching. This is especially necessary if you have many repositories and/or many page views in a short time.

CGit uses a simple hash mechanism to check if a request is already in the cache and not too old (configurable, see list below). If such a cache entry is present, it will be delivered instead of re-creating the same page (the HTTP header Last-Modified stays the same, i.e. the browser knows when the page is from).

CGit also caches the result of scan-path. This way CGit doesn’t have to add all repositories one by one for the overview page each time.

Another important factor that influences how fast the index page builds up is the use of so-called age files. The Idle column is usually recreated each time CGit goes through the branches of each repository and notes the age. This is not very fast though.

It’s more practical to use one file per repository, indicating when the last commit was uploaded. This is best done with hooks (see Sec. 8.2, “Hooks”). Use this command in the post-update hook:

If you want to use a different path instead of info/web/last-modified (relative to $GIT_DIR), use the CGit configuration key agefile for the specification.