Unix

From DreamHost

(Redirected from KB / Unix)
Jump to: navigation, search

The servers DreamHost uses to host your website all run the unix-like operating system known as Linux (specifically, Debian Linux). Some of our hosting plans allow you to get a "shell account" on our server, which allows you to access unix commands. Although it can be intimidating at first, learning even just a little bit of how unix works can really help make your hosting experience a better one!


Contents

Common Unix/Linux shell commands.

After you telnet in to your DreamHost account, you will be presented with a '$' character (which could be different if you've changed the shell for your account). This is what is known as a prompt - a place for you to type commands. In a command line interface, this means the same thing as having an open window does in a graphical interface: it's waiting for you to do something.

Note: A good rundown of Unix commands can also be found here:

http://theory.uwinnipeg.ca/UNIXhelp/

Some common tasks you might need to know (click on the task to visit that section):

Finding Out Where You Are (pwd)

First, you may want to know which directory you're in. To do this, you run the pwd command, which shows where you are. From your prompt, you would type pwd and hit return.

In response, it will give you what is known as a pathname. If you have just telnetted into your account, most likely it is something like /home/fred (where fred is your username). This means that you are in the fred subdirectory of the homedirectory. The fred directory is known as the root directory of your account.

Listing Directory Contents (ls)

Now, let's say you want to know what is inside of a given directory. To do this, type ls and hit return. This will give you a short listing of the contents of your current directory (remember, you can see what directory you are in by using the pwd command).

You can get a more descriptive list of your directory contents by typing in ls -al, which gives you the permissions for each item, the owner, and various other pieces of information.

Changing Directories (cd)

You can change from your current directory in several ways:

  • If you wish to change into a directory within your current directory, type in cd dirname and then hit return, where dirname is the name of the directory you wish to change into.
  • If you decide you wish to go back a directory, just type cd .. and hit return. This will take you back a step.
  • You can also change into any directory with the cd command by providing a full pathname of the directory you wish to go to. So, if you wanted to change into your home directory from any location on the server, you would type cd /home/fred, assuming your username is fred.

Deleting Files (rm)

To delete files, you use the rm command. For example, if you have a file in your current directory named testfile, you would delete it by typing rm testfile then hitting return. You can also delete files anywhere on the system by providing a full path to the filename, assuming you have the correct permissions to do so. Note that this operation is not reversible, so make sure you really want to delete the file in question.

Deleting Directories (rmdir)

The rmdir command allows you to delete directories, just as you have done with files. To delete a directory, type in rmdir dirname and hit return (assuming the directory is called dirname). As with rm and cd, you can provide a full pathname to the file. Note that you cannot delete a directory unless it is empty with this command, although you can do it with a variation of the rm command. Simply type rm -r dirname instead. Note that both of these actions will delete the directory, and cannot be undone.

Running A Program

There are a few different ways to run a program on the server. Generally, you should simply type in the full pathname of the program and hit return. If you are in the same directory as the executable itself, you can simply type in ./progname and hit return (assuming it is called progname). Some utilities are set up so that they can be run from anywhere, and only require that you type in their name and hit return. In fact, all of the above commands are actually programs residing on the server.

Listing Running Processes (ps)

The ps command allows you to see which executable programs you are currently running. Typing in this command and hitting return gives you a readout with various details, such as the name of the program, the process ID, and how long the process has been running.

Killing A Running Process (kill)

The kill command allows you to terminate a running process. Using the ps command explained above, find the process ID (PID) of the process you wish to kill, and then type in kill 1234 (where 1234 is the PID of the process). This will terminate the process.

Reading The Manual (man)

When you need help with a given command, you can run the man program. It provides a way to get help with all sorts of different programs and utilities on the server. To use it, type in man command (where command is the name of the command you need help with). For example, man ls would give you some detailed help on using the ls command. Although man can be somewhat cryptic and may contain far more information than you really need, it is very complete, and often proves valuable. Many utilities, including the ones mentioned above, have special features that can only be discovered by viewing their man page.

Many programs have a built in help feature, which you can access by using the --help flag. For example, typing in ls --help will give you help with the ls command. Often, this provides a less detailed but easier to understand display than the man command.

Getting The Date And Time (date, cal)

One somewhat handy feature that you can use is the date function. If you type in date from the command line and hit return, you will be presented with the current date and time. There are many ways to customize the formatting of this display, which you can find by typing in date --help instead. A similar utility, used by typing in cal and hitting return, provides you with a calendar for the current month. Typing in cal -y extends the command to provide a calendar for the entire year.

Setting timezone information for your unix shell or perl cgi scripts

tzselect is a useful command that will help you determine the correct timezone setting that you can put into your .bash_profile. You will have to logout and login again for the variable to take effect. For example, I determined that my optimal TZ setting was 'America/Toronto' 

# ~/.bash_profile: executed by bash(1) for login shells.

export TZ='America/Toronto';
export PATH=$HOME/bin:$PATH:.
umask 002
PS1='[\h]$ '

Configuring the TZ environment variable enables the date command to use the correct timezone.

So, for your perl-cgi script: Before you call the date command and store it in a variable, you would do something like this... 

my ($TZ) = 'America/Toronto';
$ENV{'TZ'}= "$TZ";
$date = `date`;

Changing Permissions (chmod)

To change read, write, and execute permissions for a given file or directory, you must use the chmod command. This command allows you to set permissions for a given file, and is often used when writing CGIs for use with your web site. For full details on many ways to use this command, you should visit http://catcode.com/teachmod/.

Generally, most CGI scripts should be set to mode 755, which you can do by typing in chmod 755 filename.cgi, assuming the file is called filename.cgi and is in the same directory.

Moving or Renaming Files/Directories

To move or rename files or directories, you should use the mv command. To move a file or directory from one directory to the other, you should type in mv source destination and hit return, where source is the file you wish to move, and destination is the directory to which you are moving it (usually an absolute to relative pathname). Renaming a file/directory is the same process, except you provide a different file or directory name for the destination.

Using A Text Editor (pico, vi, emacs)

For most people, the use of the pico text editor is sufficient for most tasks. To use it, just type in pico filename (where filename is the name or path to a file you wish to open) and hit return. This will allow you to use pico, an easy-to-use text editor. We recommend this to most of our users.

Others prefer more advanced text editors, such as emacs and vi. Although more difficult to master, these editors are powerful once you do learn how to use them. To open a document with these editors, type in emacs filename or vi filename, where filename is the file you wish to open.

Getting Information On A Domain (whois)

You can instantly get information on a given domain name by using the whois command. This allows you to see who is registered to a given domain, when it was created, who to contact at that domain, etc. This is also a useful way to see if a domain you have registered has been registered by Internic yet. To use it, simply type in whois domain (where domain is the domain name you are checking on, ie dreamhost.com) and hit return.

Various Net Services (lynx, telnet, ftp, gopher)

A command line based web browser is available for your use called lynx. Although it is obviously devoid of any graphics display, many swear by the speed in which lynx allows you to go from site to site. To use lynx, simply type in lynx http://www.sitename.com (where the URL is to a site you wish to visit) and hit return.

You can also telnet in to another server from your shell account by typing in telnet hostname, where hostnam is the name of the server you are connecting to. FTP and Gopher (for those few gopher sites still left out there) work the same way, but with the ftp and gopher commands, respectively.

Do you support ssh to access your shell accounts or just telnet?

We support and encourage the use of ssh to access shell user accounts on our servers!

We take security very seriously here and it helps us a lot if our users practice good security habits as well! Using ssh (secure shell) instead of telnet to access your account is a big big help!

Can I run any program I'd like to from my shell account?

You must be responsible in your use of the shell account.

None of your programs may attempt to breach the security of our system or any other, nor try and interact with other users.

The primary use of our server is for web serving, so we would like any scripts you run to be related to that. If you have a need for software that is not directly related to the serving of web pages, we encourage you to contact technical support first to see if it's okay.

If you have a special request, please contact support, and inform us about the program you plan on running.

Is there any way around directories being case sensitive?

Sorry but there really isn't. In unix systems, both directories and file names are case-sensitive. Domain names, however are not!

Really you should just make sure that all your links are correct, and maybe just standardize on all lower-case directories and filenames.

You COULD, if you were a stud programmer, make your 404 file actually be a cgi script which reads in the environment variable for the URL it is at, and then lower cases it, and checks if the lower-case version exists, and if so, redirects you there, and if not, outputs a 404 page..

But that'd be a lot of work!

Added: stedhwiki on 4/6/2006:

Another method that does not require it to be a CGI or PHP:

1) Add the following line to your .htaccess file: 

ErrorDocument 404 /404.htm

This will route all "failed" pages to the file /404.htm

2) Create a file called 404.htm with at least 512 bytes and including some variation of the following script: 

<script language="javascript">
  var myRoute = document.location + "";
  var myRouteLC = myRoute.toLowerCase();
  if (myRoute!=myRouteLC)   {	document.location=myRouteLC;   }
</script>

This will check to see if the current URL has any "uppercase" elements. If it DOES, then it will route the page to the lowercase version. If the url is ALL lowercase, then it will do nothing - thus displaying the content of your 404.htm.

Another convenient part of this script is that it WILL allow you to have mixed case files if you choose to, but just simply tries the lowercase version if the upper/mixed case fails. This is not the case with such things as modrewrite (which usually do not work on DH/shared servers anyway).

Hope this helps

Added by IMarvinTPA on 11/1/06:

1) Vote for the feature request to add mod_speling to the servers.

2) Use mod_rewrite to correct url case in .htaccess: 

RewriteEngine On
RewriteRule   ^dndlive/(.*)$  dndlive/$1 [nc]
RewriteRule   ^siren\.jpg(.*)$ Siren.JPG$1 [nc]
RewriteRule   ^folder/(.*)$  folder/$1 [nc]
RewriteRule   ^file\.ext(.*)$ file.ext$1 [nc]

You will need to have an .htaccess in each folder that you wish to do this to as well as a line for each file or folder in that folder. The [nc] makes the regex (the bit between ^ and $) case insensitive. The bit after that is how it will be rewritten and must match how the file is in the folder or you'll cause 404s instead of fixing them. The (.*) is a wild card and the $1 is the contents of that wildcard. You will also need to escape out any periods in your file name with a "\".

Updated by IMarvinTPA on 8/23/2009

Here is a script which will write the above in a new .htaccess file: 

#!/usr/local/php5/bin/php -q
<?php
//Recurse will follow into subdirectories.
//Watch out if you have installed any One-clicks, they may use .htaccess files.
//You will want to copy them as base.htaccess so this script can safely keep their contents.
$recurse = false;
//Overwrite will replace the existing .htaccess file with the one this script generates automatically.
//If it is false, it will leave a new.htaccess file for you to look at and use at your discression.
$overwrite = false; 
//change this to the directory you wish to start your htaccess file manipulations from.
//Generally, this will be /home/[yourusername]/[domainname]
$startdirectory = '.';

InsensitiveDir($startdirectory, $recurse, $overwrite);


function InsensitiveDir($path, $recurse, $overwrite) {
	$files = scandir($path);
	$of = fopen($path . '/new.htaccess', 'w');
	$bf = fopen($path . '/base.htaccess', 'r');

	if ($of === false) {
		die('Failed to open new.htaccess file for output.');
	}
	//Use the base file if provided.
	if ($bf === false) {
		//The base of the .htaccess file if none was provided.
		fwrite($of, "AddHandler php-cgi .php\n");
		fwrite($of, "Action php-cgi /cgi-bin/php.cgi\n");
		fwrite($of, "RewriteEngine On\n\n");
	} else {
		//Use the contents of the base.htaccess file for the start of our new one.
		fwrite($of, fread($bf, filesize($path . '/base.htaccess'))); 
	}
	fclose($bf);

	fwrite($of, "#The following lines protect the script files.\n");
	fwrite($of, 'RewriteRule ^base\\.htaccess(.*)$ index.php$1 [nc]' . "\n");
	fwrite($of, 'RewriteRule ^old\\.htaccess(.*)$ index.php$1 [nc]' . "\n");
	fwrite($of, 'RewriteRule ^new\\.htaccess(.*)$ index.php$1 [nc]' . "\n");
	fwrite($of, 'RewriteRule ^buildhtaccess\\.phpsh(.*)$ index.php$1 [nc]' . "\n");
	fwrite($of, "\n#Case Corrections\n");
	
	foreach ($files as $fnum => $file) {
		$file = rtrim($file, "\r\n");
		echo "Looking at '$file'.\n";
		if (is_dir($path . '/' . $file)) {
			if ($file != '.' && $file != '..') {
				fwrite($of, 'RewriteRule ^' . preg_quote($file) . '/(.*)$ ' . rewrite_quote($file) . '$1 [nc]' . "\n");
				if ($recurse) {
					InsensitiveDir($path . '/' . $file, $recurse);
				}
			}
		}
        else
		{
			if ($file != 'base.htaccess' && $file != '.htaccess' 
			 && $file != 'new.htaccess' && $file != 'old.htaccess'
			 && $file != 'buildhtaccess.phpsh') {
				fwrite($of, 'RewriteRule ^' . preg_quote($file) . '(.*)$ ' . rewrite_quote($file) . '$1 [nc]' . "\n");
			}
		}
	}
	fclose($of);
	//Flag to allow this program to automatically overwrite the existing .htaccess file.
	if($overwrite) {
		if (file_exists($path . '/.htaccess')) {
			//Only keep the ORIGINAL .htaccess file.
			if (!file_exists($path . '/old.htaccess')) {
				copy($path . '/.htaccess', $path . '/old.htaccess');
			}
			unlink($path . '/.htaccess');
		}
		copy($path . '/new.htaccess', $path . '/.htaccess');
		unlink($path . '/new.htaccess');
	}
	return true;
}

function rewrite_quote($mystr) {
  $mystr = Replace($mystr, ' ', '\ ');
  $mystr = Replace($mystr, '$', '\$');
  return $mystr;
}

/*
A replace function that works like VB's, basically they removed str and reordered some parameters.
*/
function Replace($src, $fnd, $rpl, $str=1, $cnt=-1, $sense=0){
	if ($sense == 0)
		return str_replace($fnd, $rpl, $src);
	else
		return str_ireplace($fnd, $rpl, $src);
}
?>

Save this file as buildhtaccess.phpsh in your websites root directory. If you have shell access, you can execute chmod 700 buildhtacces.phpsh on the shell command line. Then, to run it, type ./buildhtacces.phpsh

Please note, the default settings of this script are not destructive and will appear to have no effect. It will only do the directory you executed it from, whatever pwd returns. It does not create or replace a .htaccess file. It will create or replace a new.htaccess file.

If you have existing .htaccess files with custom rules already, you will need to rename them to base.htaccess and this will be used as the basis of the new file.

Added by theraven on 3/27/2007:

ErrorDoc method:

Make an error.php and .htaccess in the top level of the domain name (/home/users/username/domain_name) with the following:

  • Known (Unavoidable) Issues:
    • If two or more files contain the same letters (e.g. fileName1.txt and filename1.txt), the highest one will be redirected to first (meaning that in the "e.g." case, filename1.txt (1111111111111 in binary) would be redirected to since fileName1.txt (1111011111111 in binary) is less than filename1.txt).
    • Since special characters and numbers (/, ., \, |, &, =, 1, 2, 9, 0, etc.) don't have an upper/lower-case, all upper/lower-cased versions will be checked multiple times if the filename initially not found contains any of said characters or numbers (there is always at least 1 (the "/" at the beginning)).
    • If $_SERVER["SCRIPT_URL"] is too long, the script will die a horrible death (exceed max_execution_time limit).
    • The longer $_SERVER["SCRIPT_URL"] (everything after the domain name is "$_SERVER["SCRIPT_URL"]"*) is, the longer the script takes; thankfully, this script only executes if the file is not found.
    • If error.php is accessed directly, an infinite loop will occur resulting in a 500 error.
  • $_SERVER["SCRIPT_URL"]: 
<?

// Inside error.php
// This script will find the file by systematically replacing letters with their upper/lower-case counterpart.
// The first checked is the all lower-cased version.
// The last checked is the all upper-cased version.
$new_SCRIPT_URL = $_SERVER["SCRIPT_URL"];

// Create a binary string to represent all lower-cased letters.
$binary_number = str_repeat('1', strlen($new_SCRIPT_URL));

$dec_num = bindec($binary_number);

// Count backwards from $binary_number to 0.
for ($i = $dec_num; $i >= 0; $i--) {
    // Pad the binary string so it's always the same length as $new_SCRIPT_URL.  
    $binary_number = str_pad(decbin($i), strlen($new_SCRIPT_URL), '0', STR_PAD_LEFT);
    $temp = '';
    // Cycle through each "bit". If it's 0, use the upper-case and if it's 1, use the lower-case.
    for ($j = 0; $j < strlen($new_SCRIPT_URL); $j++) {
        $temp .= ($binary_number{$j} == '0' ? strtoupper($new_SCRIPT_URL{$j}) : strtolower($new_SCRIPT_URL{$j}));
    }
    // Check the filename we just made. If it's a file or folder, set $new_SCRIPT_URL equal to it and break from the for loop.
    if (file_exists($_SERVER["DOCUMENT_ROOT"] . $temp)) {
        $new_SCRIPT_URL = $temp;
        break; // No need to keep going.
    }

    // Comment out this line, if you don't want to tell the user what other files you tried.
    $attempts[] = $temp;
}
// Check $new_SCRIPT_URL again, just in case something changed and redirect.
if (file_exists($_SERVER["DOCUMENT_ROOT"] . $new_SCRIPT_URL)) {
    $redirect = str_replace($_SERVER["SCRIPT_URL"], $new_SCRIPT_URL, $_SERVER["SCRIPT_URI"]);
    header('Location: ' . $redirect);
} else {
    // If $new_SCRIPT_URL doesn't exists, tell the user that the file could not be found and also tell him what we tried.
    echo '<h1>404 Error:</h1><h2>Could not find ' . $_SERVER["SCRIPT_URL"] . '.' .

    // Comment out this line, if you don't want to tell the user what other files you tried.
    ($attempts ? '<br />Also tried:<br /> ' . implode('<br />' . "\n", $attempts) : '') .

    '.';
}

?>

# Inside .htaccess
ErrorDocument 404 /error.php

How do I set up CVS?

I want to use cvs so that two or three people can work on my web site at once. How do I get it set up on my acount?

This is for advanced users only who are familiar with the use of cvs.

You can actually set up cvs yourself from any shell account. The process is as follows:

  • Create the directory you want to be your CVS root directory:

mkdir ~/cvsroot (or whatever you like).

  • If your shell is tcsh, add this line to ~/.tcsh_profile:

setenv CVSROOT /home/username/cvsroot (or whatever your CVS root dir is).

  • If your shell is bash, add this line to ~/.bash_profile (Use pico to do this i.e. pico .bash_profile ):

export CVSROOT=/home/username/cvsroot (or whatever your CVS root dir is).

  • Run this command:

cvs init

Keep in mind that this method will set you up with a cvsroot directory and will allow other people to access it, but it does not provide a complete setup for sharing access to a website with other users. It is up to you to populate the cvsroot with whatever data you want, and it is up to you to copy the cvs files into the live website directory at the appropriate times after changes have been checked into cvs. Also note that you will not be able to view the contents of the cvs repository unless you set up a subdomain or link to the cvsroot directory from your regular website. You may also want to password-protect the cvs version of your website so it is not publicly accessible.

Note that we don't offer anonymous cvs access - just cvs for local users!

Also, here is a nifty tool for Windows users to work with CVS through their local file system: TortoiseCVS.

Retrieved from "http://wiki.dreamhost.com/Unix"
Personal tools