Crontab

From DreamHost
(Redirected from Cron Jobs)
Jump to: navigation, search
The instructions provided in this article or section require shell access unless otherwise stated.

You can use the PuTTY client on Windows, or SSH on UNIX and UNIX-like systems such as Linux or Mac OS X.
Your account must be configured for shell access in the Control Panel.
More information may be available on the article's talk page.


Overview

The crontab command, found in Unix and Unix-like operating systems, is used to schedule commands to be executed periodically.

Generally, crontab uses a daemon, crond, which runs constantly in the background and checks once a minute to see if any of the scheduled jobs need to be executed. These jobs are generally referred to as cron jobs. Cron jobs run as the user who creates them, as though that user typed the command into their shell.

Important icon.png Important: Only shell users can create cron jobs. For help setting up a shell user, see the Enabling Shell Access wiki.


Setting up a cron in the panel

You can have DreamHost automatically configure a cron job in the panel under one of your shell users.

To set up a cron in the panel:

  1. Navigate to the (Panel > 'Goodies' > 'Cron Jobs') page.
    The 'Cron Jobs' page appears:
    01 Crontab crontab page.png
  2. Click the Add New Cron Job button
    The 'Creating New Cron Job' page opens:
    950px-02 Crontab simple example.png
  3. To create the cron, fill in the following fields:
    • User: From the dropdown menu choose an existing shell user you'd like the cron job to run under.
    • Title: Give the cron job a name you'll remember.
    • Email output to: Enter an email address you'd like to receive the cron output to. You can also leave this blank to not receive emails.
    • Command to run: Enter the command to run.
    • Use locking: Enabling this if you want to prevent the job from running more than once at a time.
    • When to run: From the dropdown menu, choose how often you'd like this job to execute.
    Note2 icon.png Note: The command you enter usually must first specify the path to the program you'd like to use to run the cron job. For example let's say you have a PHP file named script.php in your domains directory:
    • /home/username/example.com/script.php

    To run this command you'd enter the path to your chosen version of PHP followed by a space, followed by the path to the file:

    • /usr/local/php55/bin/php /home/username/example.com/script.php



Crontab file contents on the server

Due to the complex nature of translating user supplied content from the panel to the server, what you enter in to the panel is not what you will see in the crontab file. The following example is a more complex cron job using a wget command.

950px-01 Crontab panel.png

The crontab file on the server will then appear as follows:

###--- BEGIN DREAMHOST BLOCK
###--- Changes made to this part of the file WILL be destroyed!
# Cron Process
MAILTO="admin@dhwiki.dreamhosters.com"
*/10 * * * * /usr/local/bin/setlock -n /tmp/cronlock.3788261262.173085 sh -c $'wget -q -O  dev/null http\072//dhwiki.dreamhosters.com/cron.php'
###--- You can make changes below the next line and they will be preserved!
###--- END DREAMHOST BLOCK

There are a couple of things worth noting:

  • http://” has been translated as “http\072//”. The “:” (colon) character is one of several special characters that need to be altered in to some other Unix-readable format in order to be successfully added to the crontab file. This is using octal notation (“\072” is the octal representation for “:”).
  • When the 'Use locking' option is selected, the command(s) you want executed are filtered through a unique invocation of the setlock command.

View the setlock man page for further information. The use of setlock will result in error messages if your cron job is invoked before the last iteration of the job releases your unique lockfile. This can happen if your script takes more time to run than the time between job executions.

Execution Environment

One important consideration is that the cron daemon was designed in such a way that it does NOT execute commands within your normal shell environment. This means you cannot use bare commands in cron the way you would from the SSH shell command line. Example:

* * * * * wp plugin update --all --path=dhwiki.dreamhosters.com

This is because the PATH environment variable is /usr/bin:/bin, and the SHELL environment variable is set to /bin/sh. The cron daemon doesn't know where the 'wp' command is located. You must specify the full path to all commands not in the default PATH, and always specify the full path for files. For example:

* * * * * /usr/local/bin/wp plugin update --all --path=dhwiki.dreamhosters.com

If you have any trouble locating where a command is, you can use the following command:

which command-name

Example:

which wp

This will print to your terminal:

/user/local/bin/wp

Using sudo and cron with an admin user

If you’re on a VPS or a Dedicated Server and have an admin user, do not use 'sudo' in your crontab. Instead of your script working, you’ll get an error like this.

sudo: no tty present and no askpass program specified

This is because 'sudo' requires a password to be entered in order to run and cron jobs (along with any program/command/script you could execute from them) do not allow any input to be specified.

If you need root level permissions for a script, you’ll need to create a file in crontab format in the /etc/cron.d/ directory. Of course, make sure to keep a backup of that file as that location is outside the /home directory and not included in DreamHost’s internal backups.

MAILTO variable requirement

For performance reasons, DreamHost now requires all crontabs to have a MAILTO variable set.

The MAILTO variable can be set to anything. If you leave it blank it will not email output of your cronjobs. You can set it to your username to have it delivered to your user as usual, or you can specify an external email address. Please note that if you specify an external email address, any cron emails that get sent (including completion/error notification messages) factor in to your shell user's hourly email sending quota. For more information about that limit, please view the SMTP quota wiki.

Note2 icon.png Note: If your cron job doesn't create any output on the command line, no email will be sent even if the cron job is set to send email.


Examples of a valid MAILTO configuration in a crontab file

To disable sending cronjob output via email:

MAILTO=""

To send cronjob output to someuser@example.com:

MAILTO=someuser@example.com

To send cronjob output to your shell user’s Maildir folder:

MAILTO=youruser
Note2 icon.png Note: As stated when running man 5 crontab, strings are not parsed for environmental substitutions or replacement of variables. This means that MAILTO=$LOGNAME will not work.


Cron job character limit

When adding/modifying a cron job in the panel, the 'Command to run' field specifies a maximum of 1000 characters. This limit does not include the additional content DreamHost adds to the cron job (such as locking or the sh -c $’’ encapsulation). On average, that additional content should not be more than 100 characters. Violating the limit results in this warning:

/bin/sh: -c: line 1: unexpected EOF while looking for matching `''
/bin/sh: -c: line 2: syntax error: unexpected end of file

If you encounter this error it’s recommended that you encapsulate your commands into a script and modify the cron job to execute the script instead. For example, if you were to place the command (or commands) that contain more than 1000 characters in a file named ‘cron.sh’ and place that file in your user’s home directory, you could then set the cron job command as follows.

. ~/cron.sh

Be sure to review the Crontab#Execution Environment section above to make sure you account for any missing variables, paths, and/or declarations.


Advanced technical details of a crontab

  • The crontab files are where the lists of jobs and other instructions to the cron daemon are kept. Each user at DreamHost has their own individual crontab file and there is a system wide crontab file at /etc/crontab (but this file can only be edited by a system administrator).
  • Crontab files are simple text files that have a particular format. Each line of a crontab file follows a particular format as a series of fields, separated by spaces and/or tabs. Each field can have a single value or a series of values. A single cron job should take up exactly one line, but this can be a long line (more than 80 characters).
  • Each line has five time/date fields, followed by a command, followed by a newline character (’\n’). A common problem is not including a newline, so hit 'Enter/Return' a time or three at the end of your command.
  • Another common problem is automatic word-wrap breaking up a long line into multiple lines, so make sure your text editor doesn't do this.
  • Blank lines and leading spaces and tabs are ignored. Lines whose first non-space character is a hash-sign (#) are ignored as they are considered comments. Note that comments are not allowed on the same line as cron commands, since they will be interpreted s being part of the command. Similarly, comments are not allowed on the same line as environment variable settings (like MAILTO).


Date/Time Fields

The first five fields of the line are the date and time field which specify how frequently and when to execute a command. When adding the cron job in the DreamHost panel, the Date/Time is added for you automatically based on your 'When to run' setting.

Field no. Description Permitted values
1 minute 0-59
2 hour 0-23
3 day of the month 1-31
4 month 1-12
5 day of the week 0-7

Note: For day of the week, both 0 and 7 are considered Sunday. The time is based on that of the server running cron.

Another (graphical) way of looking at these fields.

 # * * * * *  command to execute
 # │ │ │ │ │
 # │ │ │ │ │
 # │ │ │ │ └───── day of week (0 - 6) (0 to 6 are Sunday to Saturday, or use names; 7 is Sunday, the same as 0)
 # │ │ │ └────────── month (1 - 12)
 # │ │ └─────────────── day of month (1 - 31)
 # │ └──────────────────── hour (0 - 23)
 # └───────────────────────── min (0 - 59)

There are several ways of specifying multiple values in these fields:

  • The comma (',') operator specifies a list of values.
    • 1,3,4,7,8
  • The dash ('-') operator specifies a range of values.
    • 1-6
    • This is equivalent to "1,2,3,4,5,6".
  • The asterisk ('*') operator (frequently known as a wildcard) specifies all possible values for a field. For example, an asterisk in the hour (second) field would be equivalent to 'every hour'.
  • The slash ('/') operator can be used in conjunction with an asterisk to skip a given number of values. Example:
    • /3
    • This means to skip to every third value. So "*/3" in the hour field is equivalent to "0,3,6,9,12,15,18,21"; "*" specifies 'every hour' but the "/3" means that only the first, fourth, seventh, etc. values given by "*" are used.

You can also use one of these special strings in place of the time/date fields.

Entry Description Equivalent to
@yearly (or @annually) Run once a year at midnight on January 1 0 0 1 1 *
@monthly Run once a month at midnight on the first day of the month 0 0 1 * *
@weekly Run once a week at midnight on Sunday morning 0 0 * * 0
@daily (or @midnight) Run once a day at midnight 0 0 * * *
@hourly Run once an hour at the beginning of the hour 0 * * * *
@reboot Run at startup (of the cron daemon) @reboot

Review the following link from wikipedia for further information:

http://en.wikipedia.org/wiki/Cron#Nonstandard_predefined_scheduling_definitions

Output

The output of the cron job is determined by what is sent to the terminal as a result of the commands/script that are executed. By default, all output is emailed to the location specified in the MAILTO variable (see the Crontab#MAILTO_variable_requirement section for more information). As noted above, if your cron job command doesn't create any output on the command line then no email will be sent.

You can provide special instructions for the standard out (STDOUT) and standard error (STDERR) output by using the ">" operator. When you use ">" without a number before it, it defaults to "1>". This is the standard (non-error) output.

When you use "2>" you are specifying what to do with the error output. So, for example, ">my_file.txt" would redirect standard output to a file called "my_file.txt", and "2>my_errors.txt" would redirect the errors to a file called "my_errors.txt".

Permissions

By default, files created on DreamHost's servers have a permissions level of 644. If you choose to execute a script via cron, you may need to set the permissions for the file to 744 using chmod in order to allow it to execute properly.

Examples

  • Example 1: This will run a command at 4:10 PM PST/PDT, and email you the regular and error output to the destination specified by MAILTO.
10 16 * * * /home/username/bin/yourscript.pl


  • Example 2: This will run a command at 2:00 AM PST/PDT on Saturday, and the only output will be errors.
0 2 * * 6 /home/username/weekly/weekly-pruning.sh > /dev/null


  • Example 3: This will run at midnight on New Years Day (January 1st), and there will be no output.
0 0 1 1 0 /home/you/happy.new.years.pyc >/dev/null 2>&1

2>&1 is a special redirect that sends the standard error (“2>”) output to the same place as the standard out (“>” or “1>”) output.


  • Example 4: This will run a PHP script called cron.php at the top of every hour.
0 * * * * wget -O /dev/null http://www.example.com/cron.php


  • Example 5: This will run a local script (i.e. hosted at DreamHost) every 15 minutes.
*15 * * * * /usr/local/bin/perl /home/username/example.com/myscript.pl


CONTINUE HERE

  • Example 6: This will run an external script (i.e. hosted elsewhere) every 30 minutes using curl.
*30 * * * * /usr/bin/curl -s http://domain.com/send.php &> /dev/null

&>/dev/null is an abbreviation for 1> /dev/null 2> &1. It redirects both file descriptor 2 (STDERR) and descriptor 1 (STDOUT) to /dev/null.

View http://unix.stackexchange.com/a/70971


Manually editing Crontab

To edit the your crontab manually you'll have to log in to your server via SSH. You can replace your existing crontab with a file you have uploaded using the following comamnd:

crontab /home/username/filename

You can also directly edit your crontab by executing the following command:

crontab -e

Once you've saved your changes and exited the text editor, you can check that the change was successful using the following command:

crontab -l
  • To remove your crontab and start fresh:
crontab -r

If you are logged in as a VPS admin user, you can edit the crontab file directly. It is stored here:

/var/spool/cron/crontabs/youruser

Don’t forget you’ll need to use sudo (or start an interactive session as the root user with sudo -i) to access that file. Example (opening the file with the 'vi' text editor):

sudo vi /var/spool/cron/crontabs/youuser

Troubleshooting

  • When testing your new cron job, set the 'When to run' time frame to run every 10 minutes. Also make sure to set an email address for reporting the results. This will give you important information about what may be going wrong if the command line is not running the cron job as desired. When everything is running properly you can then adjust the 'When to run' value to anything you like.
  • If you're planning to use external software with a cron job, make sure you have set 'write' and other permissions for the files in your remote folders. If you get an error message about "permission denied" it means your permissions are not set properly. Also specifying the full path of the file you’re writing to can prevent issues with the cron environment running from a different directory.

VPS & Dedicated server troubleshooting

It's rare that the cron service on a server may stop running. If you’re using a VPS or Dedicated server, you can verify that cron is running by running this shell command:

ps aux | grep cron 

You can also check the system log for entries labeled CRON.

sudo grep CRON /var/log/syslog

There will usually be at least one CRON line per minute in the /var/log/syslog file. If you don’t see any recent entries then you may want to restart the cron service. To restart cron run this command under your admin user:

sudo service cron restart

See also