Revision as of 00:43, 4 February 2012

Git is a version control system. This page explains how you can host Git repositories on your Dreamhost account.

Note: Dreamhost does not officially support git, if you're interested in git-daemon, you might want to vote for Git support.

Pre-installed git

Dreamhost has started installing git on some servers. Before trying the schemes outlined below check first and see if your server has git installed.

[daedalus]$ which git
/usr/bin/git
[daedalus]$ git --version
git version 1.4.4.4
[daedalus]$ 

Note - if you use the preinstalled git, download the corresponding git tarball (1.4.4.4 in the example above) and get the gitweb support from that specific version. gitweb is tied to the version of git that it ships with. I had a more recent gitweb installed locally and it took me hours to figure out why it was failing.

Shared repository

If you plan to share the repository with multiple users, you may want to config it to keep the original permissions.

  git repo-config core.sharedRepository group

SSH

Setup One: For the Impatient

If git is already working on your server (it was on mine), this will get you up and running ASAP.

# Setup SSH keys
[local ~]$ ssh-keygen -t rsa
[local ~]$ ssh-copy-id [user@]host    --> OSX users: cat ~/.ssh/id_rsa.pub | ssh user@machine "mkdir ~/.ssh; cat >> ~/.ssh/authorized_keys"
[local ~]$ eval `ssh-agent`
[local ~]$ ssh-add

# Create the local repository
[local ~]$ cd project
[local project]$ git init
[local project]$ touch .gitignore
[local project]$ git add .
[local project]$ git commit

# Create a bare remote repository
[local project]$ ssh [user@]host
[host ~]$ mkdir project.git
[host ~]$ cd project.git
[host project.git]$ git init --bare
[host project.git]$ exit

# Push to the remote repository
[local project]$ git remote add origin ssh://[user@]host/~/project.git
[local project]$ git push origin master
[local project]$ git branch --set-upstream master origin/master

Note: if you want to copy the files to your web directory, you can then set up a post-receive hook as explained here: http://toroid.org/ams/git-website-howto

# Clone from the remote repository
[other ~]$ git clone ssh://[user@]host/~/project.git

Setup Two: More Thoughtful

wget a git tarball, build it, and install it. This will involve getting other things like autotool, libcurl, etc. Use this git to pull down the latest git from the main git repository. Build again but do it this time like this: make NO_MMAP=1. That will turn off git's use of mmap. Without doing this the DreamHost autopolice (procwatch) will kill your git processes all of the time because of their "excessive" memory consumption. Git processes really aren't using 500MB of memory, they just have two 250MB files mmaped. But DreamHost doesn't make the distinction and will zap the process. Make sure git is on your shell path and working.

Set up your shell account to use use ssh keys and not need a password. On your local git repo add an entry for your DreamHost repository.

 [remote "yourdomain"]
       url = ssh://youruser@git.yourdomain.tld/~/projects/yourdomain-kernel.git
       fetch = +refs/heads/*:refs/remotes/dreamhost/*
       push = refs/heads/*

Then git push your repository to DreamHost. Use a link to make sure your web server can see the repository. Use git clone to get it back with an http url: http://git.yourdomain.tld/projects/yourdomain-kernel.git

If your are working on the kernel, the set up is more complex. First clone a copy of the kernel from kernel.org. Then rename it to your repo name and push your changes to it (you'll have to futz with master). This will result in the generic kernel ending up in a giant pack file and your changes in loose objects. Now you can use this trick.

 git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git yourdomain
 cd yourdomain
 git config remote.origin.url  http://git.yourdomain.tld/projects/yourdomain-kernel.git
 git pull

Cloning from kernel.org is about 100x faster than cloning from DreamHost. Without doing this trick it takes an hour to do a clone from git.yourdomain.tld.

Of course, there are about 100 steps omitted in this explanation, but it should give your an idea of how things work. Things would be a lot simpler if DreamHost would install a copy of git and mark it so that autopolice (procwatch) doesn't keep killing it. If things don't work at first, don't give up. This configuration does work but it can take a bit of effort and persistence to get it all sorted out.

Setup Three: Gitosis or Gitolite

Git by itself, out of the box, provides no method of access control. If you know a URL for a git repository, you can clone it. Without some mechanism of access control, you can commit to it as well.

Gitosis

Gitosis is a wrapper that provides fine grained access control to your publicly visible git repositories.

You can configure gitosis such that each user has different authentication and authorization levels for each repository in the set of repositories contained in the directories of a user account.

Gitosis is no longer under maintenance.

Gitolite

There is a newer clone of gitosis called gitolite, which supports some more fine-grained access controls (such as branch- and tag-level permissions) and other features that gitosis does not. Gitolite has a fairly serious following.

The setup is similar to that required for Gitosis, so much of the directions found on that page will be relevant for setting up gitolite as well.

WebDAV

Setup One

• There is a tutorial of this setup you might want to check. (--Carlos.lima 18:59, 26 February 2010 (UTC))

Setup a WebDAV directory

Go to panel.dreamhost.com, select "Goodies" and then "Htaccess/WebDAV"...

Create a directory to store your git repository at http://yourhostname.com/git (or git.yourhostname.com/project, or wherever). Make sure to add at least one username/password so you can upload to this directory.

While the DreamHost servers set up WebDAV, proceed to the next section.

Install Git on your local machine

• Compile it yourself

wget http://kernel.org/pub/software/scm/git/git-1.6.0.4.tar.bz2
tar xjvf git-6.0.4.tar.bz2
cd git-6.0.4
make prefix=/usr all doc
su
make prefix=/usr install install-doc
exit

• or get an RPM from kernel.org
• or search for Git at rpmfind.net (don't get GNU Interactive Tools).

Tell Git who you are

git repo-config --global user.name "Your Name"
git repo-config --global user.email yournick@yourmailserver.com

Prepare a Git repository

mkdir test-repo
cd test-repo
git --bare init-db

Now tell the repository where it will live (needed since WebDAV will be updating the repository).

git repo-config remote.origin.url http://yourhostname.com/git/test-repo/
git update-server-info

(Note: the trailing '/' is VERY IMPORTANT)

Upload to WebDAV

KDE

• Point konqueror to webdav://yourserver.com/git
• Log in using a username/password you registered with DreamHost
• Copy test-repo into the WebDAV folder. You can delete the test-repo folder from your local machine as you'll clone it later (i.e. pull it back down).

Generic linux

• Install fusedav.
• Mount the webdav folder with: fusedav http://yourserver.com/git /path/to/mount/point
• Copy test-repo into the WebDAV folder. You can delete the test-repo folder from your local machine as you'll clone it later (i.e. pull it back down).

OS X

• Finder -> Connect to server
• Connect to http://yourserver.com/git
• Copy test-repo into the WebDAV folder (i.e. "git"). You can delete the test-repo folder from your local machine as you'll clone it later (i.e. pull it back down).

Locally store your username/password

• By doing this, Git won't prompt you for them every time you sync with the WebDAV repository.
• Skip this step if security is a concern.
• Edit ~/.netrc and add a line like:

machine <YOURHOSTNAME.COM> login <USERNAME> password <PASSWORD>

• For security, set restrictive permissions on your ~/.netrc file:

chmod 600 ~/.netrc

Create and push a local repository

• Initialize a repo that will store the files you actually want to put in version control (i.e. your source files). Optionally, if you already have your source repo, you could probably skip to setting the remote.upload.url. Also, you could clone the empty repo you just made (see next step and then come back), and skip to setting the remote.upload.url.

mkdir source-repo
cd source-repo
git init

• Add a file

echo "This is a file" > a.file
git add a.file
git commit -m "commit message"

• Tell this new repo about the repo on the server

git repo-config remote.upload.url http://yourhostname.com/git/test-repo/

(remember the trailing '/')

• Upload the changes

git push upload master

• If you get the error return code 22 when pushing up to the server re-configure the remote.upload.url to contain the username

git repo-config remote.upload.url http://username@yourhostname.com/git/test-repo/

• If you're still having problems, debug the WebDAV connection via

curl --netrc --location -v http://username@yourhostname.com/git/test-repo/HEAD

• You can download changes using `git pull upload master` or just `git pull`

• Some people (like me) get the following error when attempting to clone the new git repo

Initialized empty Git repository in /example/dir/path/.git/
Cannot get remote repository information.
Perhaps git-update-server-info needs to be run there?

ANSWER 1: For me I had this error because there was a problem with my ~/.netrc file, once I added the information correctly, then I stopped getting this error and I could clone the repository.

ANSWER 2: The above solution did not work for me. However, I found that if I ran git-update-server-info on my initial local repository before uploading it to the WebDAV directory, it worked. So if you find you can't clone it when you set it up, start over and do a git-update-server-info immediately after doing git --bare init-db.

Pull the WebDAV repository

• "Clone" the repository

git clone http://yourhostname.com/git/test-repo/

• Look at its history

cd test-repo
git log

(arrows to scroll, 'q' to quit)

• Add a file

echo "another file" > b.file
git add b.file
git commit -m "set commit message here"

• Upload the changes

git push origin master

• You can download changes using `git pull origin`

Setup Two

If Setup One does not work for you, there is a different way to get it working on DreamHost. This is why Setup Two(http://stevemilner.org/blog/2007/06/24/git-dreamhost/) was created.

If you have trouble with the first one, try the steps in here: http://www.kernel.org/pub/software/scm/git/docs/howto/setup-git-server-over-http.txt for troubleshooting. Especially stuff like making sure your .netrc is working with DAV by testing with curl.

Instructions

Here is a different method for using Git on DreamHost, using davfs2, fuse and git.

First, setup WebDAV on your DreamHost account for a directory. There is no need to password protect it from being viewed over http (unless you don't want people to easily see your code). You do, however, need to make sure to setup users and passwords for editing. It may take up to 1 hour for the changes to go into effect.

You must have fuse installed. You will probably have to install fuse-devel and neon-devel followed by rebuilding fuse-davfs2 rpm.

Once you have installed fuse-davfs2, mount it just like any other filesystem:

mount.davfs http://git.yourdomain.tld/git/ /mnt/git/

If you keep your code in ~/code/ as a git repo, you can clone it on to the /mnt/git/ file system:

cd /mnt/git
git clone --bare ~/code/
git-update-server-info

After doing that, you should be able to push and pull from the repo, pushing to it via the local fs, and pulling from local fs or http url. Congrats, you now have a public Git repo!

Smart HTTP

It's also possible to use the git-http-backend CGI script. Create an executable file called git-http-backend.cgi:

 #!/bin/bash
 # Replace "/dir/with/repos" with the absolute path to a directory containing your git repositories.
 GIT_PROJECT_ROOT=/dir/with/repos PATH_INFO=$SCRIPT_URL /usr/lib/git-core/git-http-backend

And add the following to your .htaccess:

 # Replace "my-git-repos/" with the URL you want on the server,
 # or delete it completely to put everything in the root.
 RewriteEngine On
 RewriteCond %{REQUEST_URI} ^/my-git-repos/(.*/(HEAD|info/refs|objects/(info/[^/]+|[0-9a-f]{2}/[0-9a-f]{38}|pack/pack-[0-9a-f]{40}\.(pack|idx))|git-(upload|receive)-pack))$
 RewriteRule my-git-repos/(.*) git-http-backend.cgi/ [L,E=SCRIPT_URL:/$1]

With this example, the repository at /dir/with/repos/hello.git would have the web address http://example.com/my-git-repos/hello.git.

Passing PATH_INFO through SCRIPT_URL is necessary because suexec prevents most environment variables from being passed.

It should also be possible to set up write access; see git-http-backend for more information.

Other Examples

• Hosting a git repository on dreamhost -- Here's how I did it. -- Dagolden
• Keeping Git Repositories on Dreamhost Using SSH and Git'n Your Shared Host On -- I used both of these tutorials that are specific to Dreamhost to get started. --Bobjkuo 02:18, 15 January 2009 (UTC)
• Private GIT repositories (on DreamHost) -- This is yet another guide describing how to setup private HTTP-accessible Git repositories on Dreamhost using Git's git-http-backend (a.k.a git's Smart HTTP protocol) with full read-write support and password protected.

Possible Gotchas

'git clone' is killed by procwatch

One day while cloning a repository to Dreamhost I got this message:

 [genovese]$ git clone ~/git/main.git/
 Initialized empty Git repository in /mnt/local/home/##########/main/.git/
 Checking out files: 3% (57/1785)
 
 Yikes! One of your processes (git, pid 4323) was just killed because your
 processes are, as a whole, consuming too much memory. If you believe you've
 received this message in error, please contact Support.
 
 
 Killed

Git cloning a large repository (or a repository with large binary files) to Dreamhost on the shared environment can cause the procwatch to kill your process (for using too much memory on a shared host - perhaps the unpacking process is very memory intensive). An easy work-around is to "manually" synchronize the repositories, that is, copy both the files and the .git directory to Dreamhost. (Also see Killed) You'll want to do something like...

• First copy all of the files that you want

 scp -r * username@domain.com:~/website/repository_name

• Copy the .git directory

 scp -r .git username@domain.com:~/website/repository_name

• then ssh into your domain

 ssh username@domain.com

• change directory

 cd ~/website/repository_name/.git/ 

• edit your config file to make sure the remote origin section points to the right place.

 vi config

• finally, run a "git status" just to check if everything is ok.

 cd ..
 git status

Note that this principle would work with Subversion or CVS and maybe even some other SCM programs I'm not use to; you'll just need to change the specifics of the editing and command line options ('svn status' instead of 'git status', 'scp -r .svn' instead of 'scp -r .git', etc.)

Here's a (mostly) copy/pastable command list for the above. Just fill in the capitalized parts with your actual information.

 #  How to set up a git repository on DH for your 'staging' branch
 #  (which is assumed to exist already)
 #
 #   On my local machine
 $ cd ~
 $ mkdir staging.git
 $ cd staging.git
 $ git --bare init
 $ cd ~/MY_PROJECT_DIR
 $ cd checkout staging   #  Move to the 'staging' branch of my project
 $ git remote add stagingserver ssh://MY_LOCAL_USER@localhost/home/MY_LOCAL_USER/staging.git
 $ git push stagingserver staging
 $ cd ~/
 $ tar -cjvf staging.tbz2 staging.git
 $ scp staging.tbz2 MY_DH_USER@MY_DH_HOST.dreamhost.com:~/
 $ ssh MY_DH_USER@MY_DH_HOST.dreamhost.com
 $ tar -xjvf staging.tbz2   # Remember, this is on DreamHost now
 $ logout
 $ cd ~/MY_PROJECT_DIR      #  Back on my local machine
 $ emacs .git/config
 Change this line:       url = ssh://MY_LOCAL_USER@localhost/home/MY_LOCAL_USER/staging.git
 To this:                url = ssh://MY_DH_USER@MY_DH_HOST.dreamhost.com/home/MY_DH_USER/staging.git

Done! Now you can simply do "git push stagingserver staging" from your local ~/my_project directory, and watch the git'y goodness happen.</nowiki>

Another option to try is (worked for me): ssh into dreamhost and set

 git config --global pack.windowMemory "100m"
 git config --global pack.SizeLimit "100m"
 git config --global pack.threads "1" 

This limits the resources git uses and the procwatch daemon doesn't kill git anymore.

Managing Web Projects with Git

This wiki explains how to host git projects on Dreamhost. It does not explain a rational method for using git to manage your code for DH hosted sites. This page describes a method that should work quite well on Dreamhost. http://joemaller.com/2008/11/25/a-web-focused-git-workflow/

Extensions