Snapshot data restoration
From DreamHost
Contents |
[edit] Purpose
You may from time to time find it necessary to restore files or entire directories from our on-line backups (A.K.A. the "snapshot" directories). This tutorial will teach you how.
[edit] Important Notes!
NOTE1: These backups are provided as a courtesy and are NOT guaranteed to be available! If you value your data we always recommend that you maintain your own off-line backups! See these Wiki articles on how to do that. Backups
NOTE2: Sometimes it becomes necessary to drop/delete one or more snapshot backup directories if disk space is consumed at a faster rate that we anticipate. If you find that one or more of your snapshot directories are missing - this may have been the case. Do not panic! This is normal. Typically the oldest backup directory is dropped first. Missing snapshot directories will be recreated automatically as time/space allows.
NOTE3: Newly created accounts will not have any directories under the snapshot directory. This is because they have not been created yet. After about one hour you will see the first one created. The others will follow on the prescribed schedule.
NOTE4: Snapshot directories are read-only; you cannot modify them. If you try to delete or modify files or directories you will receive an error message.
NOTE5: You cannot copy files or directories from the snapshot directories on top of (replacing) the original files under your user account! This is by design. You must first DELETE file files or directories that you want to replace before coping them from the snapshot directories!
NOTE6 : We do NOT recommend using our WebFTP client to copy files or directories from the snapshot directories! Due to its various resource limitations it will most likely fail leaving you with an incomplete copy. We recommend using shell commands to delete the original files and perform the copy process as you will not run into those resource limitations. If you're not familiar with using the shell please review these Wiki articles for more information. [Unix Commands]
Now that all of those pesky notes are out of the way, we can continue with the tutorial.
[edit] Synopsis
Step 1 - Determine what files/directories need to be restored.
Step 2 - Determine how far back you'll need to go to find a good older copy of the files/directories you're looking for.
Step 3 - Restore the files/directories from the appropriate backup directory.
Step 4 - Verify that the restore was successful.
If you have accidentally deleted a file or directory then step one will be pretty easy. If the file or directory existed over one hour ago they you can restore it from the last hours backup directory.
If some kind of upgrade went wrong or your site was hacked (God forbid!) and you're not sure what files were modified then restoring the entire hosting directory from a previous backup may be the best way to go.
For whatever reason you need to restore files or directories from a backup you'll need to verify which backup contains a good copy of the files or directories you want to restore.
[edit] Availability
Our shared hosting platform makes backups automatically of your web site data (with few exceptions like symlinks to local volumes; ie: FrontPage Extension enabled domains).
Customers with our SBE "Strictly Business Enhanced" hosting plan and our DreamHost PS (Private Server) add-on service also work the same as described here.
NOTE regarding Private Servers: As of Dec. 27, 2007, attempting to enter the .snapshot directory (using cd ~/.snapshot from a shell account) will return this error: -bash: cd: /home/[username]/.snapshot: Operation not permitted. This is a known issue with Private Servers caused by a kernel level bug; however, the .snapshot backups do exist, but you must ask Support to perform restores for you for now. The plan is to resolve this within "the coming weeks" but no firm date has been confirmed. (Please remove this note when this is functioning, as well as the copy at Automated domain snapshots.)
NOTE: Our (“Half Moon”, “Full Moon” & “New Moon”) dedicated servers employ a completely different backup strategy. They do not have “snapshot” directories. Their backups are done on local hard drives.
NOTE: If you attempt to access your snapshot directories and you find that they are all empty, 0 length files or non-existent please contact support for assistance.
[edit] Snapshot Directory Structure
Our system creates a hidden directory structure under each FTP/shell user account that you create. The directory name is ".snapshot". Notice the dot (.) in front of the word "snapshot". Unix systems do not normally display directories or files with a leading dot in their names - making them "hidden". Although this is not a normal hidden directory! Some FTP clients can be configured to show hidden directories but this one won't show up because it's special. In that case you'll have to use the "cd" (change directory) command to go specifically to that directory to view its contents. More information on that later.
Here's the directory structure you'll find under the ".snapshot" directory;
.snapshot |-- hourly.0 (your files as they were one hour ago) |-- hourly.1 (your files as they were two hours ago) |-- nightly.0 (your files as they were one night ago) |-- nightly.1 (your files as they were two nights ago) |-- weekly.0 (your files as they were one week ago) `-- weekly.1 (your files as they were two weeks ago)
Some file servers may store their snapshots in this alternate format instead (year,month,day,time);
.snapshot |-- 20070607.184116 (your files as they were at this date/time) |-- 20070608.184116 (your files as they were at this date/time) |-- 20070609.184116 (your files as they were at this date/time) |-- 20070610.184116 (your files as they were at this date/time) |-- 20070611.184116 (your files as they were at this date/time) `-- 20070612.184116 (your files as they were at this date/time)
[edit] Snapshot Data Restoration Tutorial Using WebFTP
Probably the easiest way to accomplish step 1 (Determine what files/directories need to be restored from backups) is to use our WebFTP client to view your files and what's in the snapshot directories. Of course you can also use whatever other FTP client that you are comfortable with or even the shell commands if you like. Examples will be given using the WebFTP client. But the basic instructions will apply to whatever FTP client you choose to use.
NOTE: Very few (if any) FTP clients provide a local "copy" function. Therefore you will not be able to copy files from the snapshot directories without downloading them to your local computer then uploading them again. However, the WebFTP tool does have a local copy function. But it does have some serious limitations which is why we do NOT recommend using it to copy files!
Log into the web control panel at [Control Panel] and in the left-hand menu column go to [(USERS > MANAGE USERS)]. If you have more than one sub-account accessible to you, make sure that you select the proper account as the "Active Account".
Click on the "WebFTP Login" link for the user account you want to access. In this example I'm logging into the user account "dhuser".
Enter that users FTP/shell username and password (the username may already be filled in for you). Note that the login page may not always look like this. See the example below.
Once you have logged in you can click on the links for the files and directories to navigate where you want to go. As in this example.
For example, the domain I am hosting under this account is "dhexample.com". All of the web files for this domain are located under the "dhexample.com" directory. So I'll click on that directory name to enter it and view its contents. The contents are listed below.
This domain has a WordPress blog installed in its root directory (dhexample.com) so you'll see all of the normal WordPress files and directories here.
If you had a Media Wiki or phpBB site installed in other sub-directories they may be listed there as well. You'll need to get to know the directory structure of your account so you'll know your way around.
For this example, let's say that I've made a mistake editing my "wp-config.php" file. This is the main configuration file for my WordPress installation. To view this file click on the "view" button at the right side of the screen.
The contents of the file will be displayed for you to review. See the example below. Notice on the 8th line that my database password is "BAD-PASSWORD". This is incorrect and is causing my web site to give an error. But I'm sure I had the correct password before. So my goal is to see if I can replace this file with a copy from one of my snapshots where it was correct. That would get my site on-line again. Now that we know what is wrong - we can search for an older file in the snapshots to replace it with.
To go back to the file and directory listing click on the blue left arrow icon at the top-left.
Proceeding to step 2 (Determine how far back you'll need to go to find the file you're looking for).
In the large text box above the file and directory listing (where it probably contains "/dhexample.com" REPLACE that with this text ".snapshot" and press the <ENTER> key. See the example below. Below the text box it always shows what directory you are currently in. Clicking on one of those links will take you directly to that directory. In this case only "root" is clickable. But we don't want to go there now.
You have now entered the ".snapshot" directory. See the example below. It should contain directories for the various backup directories that have been created for your account. As you can see this account does not have a weekly.1 directory (it's not old enough yet). It shows up as a 0 size file instead. Also notice that the date on the other directories are all Jun 12 12:24. You can't use these dates to determine the age of the files and directories that they contain! Snapshots just don't work that way.
Click on the "hourly.0" directory to view its contents. See the example below.
These are all of the files and directories, as they existed one hour ago. Now click on the "dhexample.com" directory to view its contents. See the example below.
Click on the "view" button for the "wp-config.php" file to view its contents. See the example below.
Eureka! There's the correct database password I was looking for "NLTg7Gex". Now lets replace the current "wp-config.php" file with this on from the snapshot directory (/.snapshot/hourly.0/dhexample.com). Now if this file still had "BAD-PASSWORD" rather than what I was looking for you could check in each of the other (older) backup directories to see if there is a good copy there.
Click on the blue left arrow icon at the top left to return to the file and directory listing.
NOTE: Because this file is only 993 bytes in size the WebFTP client should have no problems coping it. So we'll do that in this example. I'll also include instructions using the shell commands as well because that's what you'd have to do for larger files and/or coping entire directories!
On to step 3 (Restore the files from the appropriate backup directory).
If you remember note6 (you do don't you?!) you can't just overwrite the existing file. You'll have to delete it first. Then you can replace it with the copy from the snapshot. So we'll do that next in our example.
Under the text box where you entered ".snapshot" in a previous step you'll find some links that allow you to go back up the directory structure quickly. Click on the "root" link now. It will take you back to the root level of this user account (its "home" directory).
Go into the "dhexample.com" directory by clicking on its link.
Click on the check-box to the left of the file you want to delete "wp-config.php". Then click on the "delete" button at the top-right. See the example below.
You will be presented with a confirmation page listing what you have selected to delete and the option to go back blue left arrow or continue green check mark. Click on the green check mark in this case.
Another confirmation screen will appear giving the results of the previous command. If there were any errors they would be listed here. Note that errors may be intermingled with success messages and may not be obvious! Another reason why using the WebFTP client may not be a good idea for restoring files from the snapshots! See the example below.
In this case the deletion was successful (there are no errors listed) so we can continue. Click on the blue left arrow to continue.
Now we need to go back to the appropriate snapshot directory to copy the file we found to replace the one we just deleted. In the directory text box enter this text "/.snapshot/hourly.0/dhexample.com" and press <ENTER>. That will take you directly back to the one hour old directory where we just were. See the example below.
Check the check box to the left of the file "wp-config.php" then click on the "copy" button at the top-right.
Another screen will appear where you can specify the location where you want the selected file(s).
Since this file is in the "dhexampe.com" hosting directory, that's where we want to copy it to. Near the bottom of that page there is a text field labeled "Target directory:". By default it contains the current directory where you just were. Using your mouse select this text ".snapshot/hourly.0/" and delete it. That should leave this text in that field "/dhexample.com". This is the directory where we can to copy this file. See the example below.
If you're coping multiple files you can also do this same step in the text box near the top next to the button labeled "Set all target directories". Then click that button and it will fill in all of the "target directories" text boxes below.
Then verify that the settings are correct and click on the green check mark icon to continue.
Another confirmation screen will appear giving the results of the previous command. If there were any errors they would be listed here. Click on the blue left arrow icon to go back to the file and directory listing. See the example below.
Now on to step 4 (Verify that the restore was successful).
Go back to the "dhexample.com" directory under the account root (as described in previous steps).
Click on the "view" button for the file "wp-config.php". Ah, success! The file has been replaced with the one from the snapshot that has the correct database password! Click on the blue left arrow to go back. See the example below.
Now the final test, visit that web page (http://dhexample.com) and verify that it's actually working, as it should. And it does! We've done it! See the example below. We have successfully recovered that file from our snapshot backups!
Now you can log out of the WebFTP client (unless you want to do anything else with it). Click on the red power button icon at the top-right corner to exit your session. On the next page you have the option to log back in again or close that window.
If you wanted to replace the entire "dhexample.com" directory the process is almost the same. Except that the WebFTP client would almost certainly fail due to its resource limitations (file size and execution time-out). Either of those could cause it to fail. That's why we recommend using the shell commands to copy files and directories from the snapshots.
[edit] Snapshot Data Restoration Tutorial Using Shell Commands
This tutorial assumes you already know how to access your shell user accounts using ssh and know basic shell commands. If not you may want to review these other Wiki articles. SSH Unix_commands
If your user accounts are currently configure to allow FTP access only then they will need to be reconfigured to allow shell access. See this Wiki article for instructions on how to do that. Enabling_Shell_Access
Here are the examples using shell commands. The procedure is almost exactly the same as outlined above. But of course it involves more typing. Be sure to take advantage of your shells "tab completion" function whenever you can! See this Wiki article for more information. [Tab Completion]
Step 1 (Determine what files/directories need to be restored from backups). Let's take a look our files.
Log into the account you want to deal with. In my example it's user "dhuser" on server "renault.dreamhost.com". The default command prompt is "[renault]$ ". See the example below.
To get a detailed directory listing type the command "ls -la" at the prompt. The "-la" option gives a long listing (with the file permissions, ownership, size, date/time) and also lists hidden files. See the example below.
Change directory into the "dhexample.com" directory (where my site is hosted) by typing this command "cd dhexample.com". The directory listing scrolls by more than filling the page. But the file I'm interested in is still on the page "wp-config.php". See the example below.
To edit/view that file we'll use the nano editor, but you could use any other editor that you're familiar with. This one is easy to use and works well for examples. Type this command "nano wp-config.php". See the example below.
Nano will give you a screen-oriented display. Notice the password "BAD-PASSWORD" on line 8; this is incorrect. See the example below. But I'm sure I had the correct password before. So my goal is to see if I can replace this file with a copy from one of my snapshots where it was correct. That would get my site on-line again. Now that we know what is wrong - we can search for an older file in the snapshots to replace it with.
To exit the nano editor press the <CTRL> + "X" keys (noted on screen as "^X" at the bottom left corner of the screen. Afterwards the command prompt will return.
Proceeding to step 2 (Determine how far back you'll need to go to find the file you're looking for).
To enter the snapshot directories type the command "cd .snapshot". Then type the command "pwd" (print working directory) to see what directory you are currently in (to get your bearings). In this case we are in the snapshot directory just below the dhexample.com directory. See the example below.
Type the command "ls -la" to view the contents of this directory.
You have now entered the ".snapshot" directory. See example below. It should contain directories for the various backups that have been created for your account. As you can see this account does not have a weekly.1 directory (it's not old enough yet). Also notice that the date on the other directories are all Jun 23 20:44. You can't use these dates to determine the age of the files and directories that they contain! Snapshots just don't work that way.
Change directory into the one hour old backup. Type the command "cd hourly.0". See the example below.
Type the command "ls -la" to get a detailed directory listing. As you can see the "wp-config.php" file is listed here. See the example below.
Type the command "nano wp-config.php" to view the contents of this file. Eureka! There's the correct database password I was looking for "NLTg7Gex". See the example below.
To exit the nano editor press the <CTRL> + "X" keys (noted on screen as "^X" at the bottom left corner of the screen. Afterwards the command prompt will return.
Let's get our bearings again. Type the command "pwd" to list the current working directory.
On to step 3 (Restore the files from the appropriate backup directory).
Now lets replace the current "wp-config.php" file with this one from the snapshot directory (/home/dhuser/dhexample.com/.snapshot/hourly.0).
If you remember note6 (you do don't you?!) you can't just overwrite the existing file. You'll have to delete it first. Then you can replace it with the copy from the snapshot. So we'll do that next in our example.
To remote the original file type this command "rm ~/dhexample.com/wp-config.php". In this case the tilde character (~) is the users root/home directory. This saves you from typing out the entire directory.
See the example below.
To copy this file from the snapshot directory to the original location type this command "cp wp-config.php ~/dhexample.com/". This will copy the file from your current location back to the original web hosting directory. See the example below.
Now on to step 4 (Verify that the restore was successful).
Type this command "cd ~/dhexample.com" to go back to the "dhexample.com" directory under the account root/home.
To view the file type the command "nano wp-config.php". Ah, success! The file has been replaced with the one from the snapshot that has the correct database password! See the example below.
To exit the nano editor press the <CTRL> + "X" keys (noted on screen as "^X" at the bottom left corner of the screen. Afterwards the command prompt will return.
Now the final test, go to that web page (http://dhexample.com) and verify that it's actually working, as it should. And it does! We've done it! We have successfully recovered that file from our snapshot backups!
[edit] Additional Examples
As an additional example, if you need to replace an entire directory here's how to do it. Once you've verified that the directory you want to replace is available in one of your snapshot directories you'll need to delete the original directory first. In this example it's the "dhexample.com" directory.
From that directory I go back up one level by typing the command "cd ..". NOTE: The double-dot file (..) represents the directory level above your current location. You can't remove/delete a directory that you are currently in.
Type the command "rm -R dhexample.com/" to remove that directory (and all files/directories it contains recursively, that's what the option "-R" means).
To copy that directory from one of the snapshot directories type this command "cp -Rv .snapshot/hourly.0/dhexample.com/ .". Note the final (.) in that last command! The single-dot file (.) represents the current directory, where we just deleted the original directory. You're essentially coping the directory from the (.snapshot/hourly.0) snapshot to the users root/home directory (where you are now). You could also use the command "cp -Rv .snapshot/hourly.0/dhexample.com/ ~" instead. In fact that may even be a little safer to use. The "-Rv" option means it will copy all files and directories recursively and "v" portion means it will list all files/directories as it goes (verbose). See the example below.
Here is the end of the copy command, with the files/directories source and target listed as it was processed. Once your command prompt returns it's done. See the example below.
Then you just need to test to verify that you've accomplished what you set out to do. That the files/directories recovered from the snapshot are good or if you still have more work to do.
[edit] Additional Help Is Available
If you have determined that you need a file or directory copied from one of your snapshots, but you find that you can't copy it because of an ownership problem you can ask support to do it for you.
Please submit a support request via our control panel at [(SUPPORT > CONTACT SUPPORT)] and give us the following information;
Use "Snapshot data restore assistance" as the subject of your support request.
1. Give the FTP/shell user account name where the files are located.
2. Give the server name.
3. List the file or directory that needs to be recovered.
4. List the particular snapshot directory from where the file or directory should be recovered (ie: "nightly.0", "nightly.1", "weekly.0").
NOTE: We can only process support requests for user accounts that are owned by the customer submitting the support request (for security reasons).
NOTE: We try to answer all support requests in 24 hours or less, usually in much less. However, there may be times when it may take over 24 hours to respond. If you specify you want a restore done from hourly.0 or hourly.1 snapshots their contents may have changed before we could process your request! Snapshots are updated CONSTANTLY! For example, if you have a corrupted file in your users account now, but the snapshot hourly.0 has a good copy. After an hour that hourly.0 backup will be replaced with the corrupted file! That good copy will be moved to the hourly.1 directory. The same thing happens with the other snapshot directories. They are constantly being updated. It may be that by the time we can process your request the data in the snapshot may have changed. We cannot be responsible if the files or directories replaced from the snapshot are not what you expected.
[edit] Conclusion
I hope this tutorial has been helpful to you. I would recommend creating a special directory and put some files into it and use it for testing. Modify the files and see how the snapshots update. Try restoring files from the snapshots so you get comfortable with the procedure. If the time ever comes when you need to restore files or directories from your snapshots you can be confident that you know what you are doing and can do it correctly. Happy DreamHosting!
