Rsync

From DreamHost
Jump to: navigation, search
See also: Rsync Backup

About rsync

rsync is a fast and versatile file copying/syncing tool. In the context of a webhost, it is particularly useful for both backing up your hosted content to a local (or another remote) drive, as well as for pushing your local content up to the webhost.

This page will focus on the common use case of pushing locally-changed files up to the webhost. See Rsync Backup for the opposite case of pulling from the webhost.

Using rsync to publish your files

rsync is particularly well suited to quickly publishing your changes to your webhost, since it is smart enough to determine which files have changed, and upload only the changes, which makes the whole "publish" operation extremely fast.

It is also possible to set filters for which files it will include and which it will ignore, which makes it possible to have a completely automated one-step setup, once your filters are in place (and once you set up ssh key authentication, so that you don't have to enter your password every time).

In the simplest case, without any filters, here's what an rsync "push out to webhost" command looks like:

rsync -e "/usr/bin/ssh"  --bwlimit=2000 -av local_directory username@server.dreamhost.com:remote_directory

where

  • username is your shell username.
  • server is the name of the machine which hosts your shell account. (eg. jezebel, cortes, etc.)
  • local_directory is the local directory in which you store the local copy of your content
  • remote_directory is the directory on the server that you wish to push to.

To avoid any mishaps, it is recommended to use the full path for both local and remote directories.

To add some filters, the easiest thing to do is to just throw them into the commandline. So, for example, you might do the following:

rsync --filter '- *.pyc' --filter '- /.git' -e "/usr/bin/ssh" --bwlimit=2000 -av local_directory username@server.dreamhost.com:remote_directory

The above ignores any *.pyc files (compiled python files), as well as the .git directory (used by Git to store the code history).

For further explanation about commonly-used rsync options, see the Rsync Backup page. For fine detail about all features of rsync, see the rsync man page.

Note: the rsync backup and rsync publish (this page) should probably be merged into a general 'rsync' page, since there's a lot of overlap in the content

Rsync for Windows

Rsync is available for Windows as well, but setup typically includes jumping through a lot of hoops. One solution that includes a GUI wrapper as well as the required Cygwin and command line rsync is Delta Copy. The 'raw' version with no installer works fine.

One note about rsync for Windows is that you use *nix style paths even if they're on your Windows machine.

To get a remote directory listing via rsync so you can get the remote path correct, use:

rsync -e ssh user@yourdomain.com:/path/

If you want to create a .BAT file to automatically sync a directory tree, try something like this:

c:\path\to\rsync -e rsync/ssh -av --delete /path/to/source/files/even/on/windows username@yourdomain.com:/path/to/destination/
pause

You may have to fiddle with the "-e rsync/ssh" since that appears to be path oriented too and seems to relate to where your "ssh.exe" executable is located. ssh.exe comes with DeltaCopy. In the first example that just has 'ssh', ssh.exe was in the same directory as rsync. In the second example, it was in a subdirectory called 'rsync'.

Another thing to note, since rsync is based on cygwin and uses *nix style paths even on Windows is how you access different Windows drive letters. If you use just '/', you'll get the root folder of the current drive or possibly the one rsync resides on. To access another drive, precede it with '/cygdrive/' like so: '/cygdrive/d/somedironDdrive/'.