Personal Backup
From DreamHost
It is against DreamHost policy for you to use your DreamHost file storage space on a web server for personal file storage that is not related to one of your websites.
There is a new feature (as of October 2008) which allows you to backup up to 50 GB of your personal files on DreamHost's dedicated "backup" servers, with the following limitations:
- it's a separate ftp (or sftp) user on a separate server
- it can't serve any web pages
- There are no backups kept of these backups (they should already be your backups, not your only copy)
If you go over 50GB, extra space is 10 cents a GB a month
Technical notes:
- To enable this service, you need to set up a Backups User Account in the DreamHost Web Panel
- This user does not have a shell on the server
- The following commands can be invoked on the server: scp sftp rsync
- Bazaar, maybe others, can push changes over sftp w/out a client on the server. I haven't tried it yet, but is there any reason why it wouldn't work for this?
Contents |
Automated Backups
This section is for advanced users and requires some technical knowledge!
To really automate backups, you'll need to set up password-less login, but modify a couple of the steps. First, you'll need a .ssh directory, but you can't upload it. You can sftp into your backup account and do: mkdir .ssh. Now you need to upload an authorized_keys file. Take a copy of your public key, rename it to authorized_keys and upload it into the .ssh directory you created on your backup account. If you want to backup from multiple machines, you'll need to create an authorized_keys file that contains all of the public keys, and then upload that. You won't be able to cat new keys onto the existing file.
Personal backups can be automated using Rsync Backup, a Unix program that supports differential backups (backing up only data that has changed since the previous backup). Windows users can install Cygwin, a Unix environment and bash shell for PCs. Another option is to modify DeltaCopy, which makes Cygwin and Rsync configurable in Windows, to run on a schedule (daily, weekly, etc.).
Also try a fantastic program called Backup-Manager for automated backup. Once configure, it will tar, zip, encrypt and upload on a schedule, automatically. See the web site for it's many, many great features.
The backup directory can also be mounted locally with SSH Filesystem and accessed with the same flexibility as your own file system. Another trick to work on top of sshfs: mount an encrypted file system inside the sshfs mount point with EncFS, which will provide a transparent layer of encryption.
How to efficiently back up multiple incremental snapshots
Rsync is easy to use for creating a single mirror of your personal files. However all local changes (including accidental file deletions or modifications) will be propagated to the remote server overwriting your old files on each backup. That defeats the purpose of having a backup in the first place. Better would be a series of snapshots of your data taken at regular intervals, but storing multiple full copies of your data is inefficient since most of the data won't have changed significantly and you'll have to transfer more than just the "deltas" which defeats the purpose of using rsync in the first place (you might as well just use sftp). Fortunately, using hard links it's possible to create multiple snapshots that take up essentially no space except for the files that have changed. Furthermore, after the initial transfer you won't have to transfer more than the "deltas" so backups will be quick and painless. This ingenious method is described in excellent detail by Mike Rubel on his rsync snapshots page. Unfortunately Dreamhost Backup users do not have shell access to execute the "mv" and "rm" commands required to implement this solution. Fortunately it's possible to duplicate the "mv" and "rm" functionality required by combining some rsync and sftp capabilities. This clever work-around is described here.
CyberDuck access
By default the Mac FTP client Cyberduck has a default Path of / - to access your files you'll need to remove that slash (in your bookmarks choose More Options and clear the Path field).
Software
Software that can be used with Personal Backup
Windows
- Cobian Backup (Open Source)
- CWRSYNC Is a packaging of rsync, ssh client and the required cygwin libraries for Windows. It runs on all versions of Windows.
- SyncBackSE (Not Free) works with FTP but not directly with secure FTP or SCP. However, you can do this by combining with Tunnelier and using Tunnelier's FTP host option that provides a local FTP service that will forward to a secure remote server.
-
rdiff-backup (Open Source) uses rsync and ssh, use with cwrsync on Windows. It is able to keep unlimited versions of backup so that you can restore from any point in time.rdiff-backup cannot be used with dreamhost backup server. It must be installed on both the client and server (and it isn't).
GNU/Linux
- Backup-Manager
-
rdiff-backup (Open Source) uses rsync and ssh. It is able to keep unlimited versions of backup so that you can restore from any point in time.rdiff-backup cannot be used with dreamhost backup server. It must be installed on both the client and server (and it isn't).
