Junk Mail

Junk Mail is a folder in the web-based SquirrelMail e-mail application. It contains messages caught by SpamAssasin .

Enabling Junk Filter

By default, DreamHost allows all messages to reach your inbox. If you would like junk mail messages to be filtered, you must enable the junk filter using the Anti-spam area of the Mail Control Panel. Please note, there are some restrictions (more info at Anti-spam ) :

• if you have catch-all email address, you won't be able to use our junk filter on that domain
• you can only turn on the filters for the whole domain, not for individual email addresses

NOTE: you are essentially changing MX records for your domain when you make this switch. As with all DNS records, the change can take up to 4 hrs to propagate. If you're hosting your name servers elsewhere, please remember to update the records at that other provider. You will be able to get your new MX records, just like any other DreamHost DNS records specific to your domain at Manage Domains area of the panel, by clicking the DNS link next to the domain name.

Accessing your Junk Mail

To protect legitimate mail, novice users can follow these three basic steps:

1. 
   • Go to https://webmail.YOURDOMAIN.com/ (.net .org, etc.) for the secure version of your Webmail (replacing, of course, "yourdomain" with your own domain name). Enter only the first part of your e-mail address (the part before the "@" symbol) and your password.
   • Users are recommended to bookmark their Webmail URL.
   • To access a non-secure version , remove the "s" from the "http" in the Webmail URL: http://webmail.YOURDOMAIN.com/
2. 
3. 
   • To read any of these messages, you have to move them to your inbox. If you’re not sure about a message, choose just the “Move to INBOX” option to read it. If clearly legitimate e-mail is found, you'll want to choose the option “Move to INBOX + Add Sender to Whitelist.” A Whitelist is a list of e-mail addresses you always want to get through the filter.

Notes

• It's a good idea to delete the real junk, because it will lessen the amount of mail to sort through the next time you check the folder.
• Note that these messages are not listed according to date, but in the order of their "Spam factor" scores, from lowest to highest. This means that if legit messages have been caught here, they will probably be near the top of the list.
• Depending on the frequency you chose, you'll get an e-mail titled "Junk Mail Report" that lists any messages you currently have in your Junk Mail folder. Be sure to read this message to see if anything legit has been blocked. (You can, of course, check this any time you like by going to Webmail.)
• If you have questions about any of this, or would like your settings changed, please read on.

Changing the Junk Mail Settings

(For more tech-savvy users)

You can change your settings in two places: 1. In Webmail, under Options → SpamAssassin Configuration or by clicking on Junk Mail and the Settings. Note: you can only change to IMAP folder filtering in Webmail) 2. In the panel, on the same Anti-spam page where you enabled the filters.

• Set up whitelists and blacklists, using individual email addresses or full domains/subdomains. You'll see examples there too. Please note: the 'From' address is not always the real sender address. Look for 'envelope sender' in the full headers, or if the message is still in the 'Junk Mail', click on the subject to open the message and see the real sender in ( ) next to the 'From' address. When an address is whitelisted, it will bypass the filters and goes directly to the inbox. When it's blacklisted, it goes directly to the junk folder, regardless of how much it may score. Please note, even if you're using IMAP filtering (sending your junk mail to an IMAP folder instead the original 'Junk Mail', the blacklisted mail will still go to the 'Junk Mail'.

- Please note, the max number of combined entries for blacklist and whitelist is 2000. Here are some tips how to avoid going over that limit: If you find yourself adding many addresses to the blacklist from a certain domain for, consider blacklisting the domain itself, and whitelist only the addresses you wish to receive mail from on that domain. If you're adding a lot of addresses in general to the blacklist, consider lowering your filtering limit: the lower that is, the more junk you catch, and reduce the need of individual blacklisting.

• Change the tag and quarantine values. Spam Assassin analyzes the email and places a score in the header. Your tag and quarantine numbers will be compared to that score, and the mail will be handled accordingly: delivered to inbox or placed in the junk folder. If you're using the original filtering method, the mail will be handled based on the quarantine limit (unless it's black or whitelisted). Tag limit will be ignored. If you switch to IMAP filtering, the tag limit will determine if the message will be delivered to Inbox or the designated IMAP junk folder (again, that is unless the sender is black or whitelisted). When using this option, the quarantine number is disabled.

The default tag/quarantine level is 999/4. That means that the messages are never getting tagged (it's highly unlikely to score a 999) and anything that scores a 4 or above will be moved to the Junk Mail folder. The lower you set your quarantine number, the higher the chance to catch the junk. However, if you go too low, you may end up with a lot of false positives. It's good to experiment a little before settling with a number.

Once you settle with a low tag or quarantine number, don't forget to check the junk mail folder (either the original, displayed under your own folder list in Webmail, or the IMAP folder you designated for IMAP filtering) from time to time, and add to your white- and blacklist, to make the filter better suit your needs.

Tip: if your email address is on a filtered domain, but you'd rather not mess with filtering and checking, just set your tag and quarantine numbers to 999/999 - they will be left alone.

• You can also set the frequency of junk reports delivered.

NOTE: once the mail is deleted from your junk folder, it's not recoverable any more! Please be sure to customize your settings, and check the junk folder from time to time to check for false positives.

Analyzing Full Headers

If you think your junk mail settings are not working as they should be, checking the headers is the first thing you should do. Most often than not, you'll find that the message wasn't sent to or from the address you exptected. Here is a sample header:

Return-Path: <newsletter@groovetickets.com> 
X-Original-To: me@mydomain.com 
Delivered-To: m6025572@spunkymail-mx8.g.dreamhost.com 
Received: from enforcer.dreamhost.com (sd-green-bigip-177.dreamhost.com [208.97.132.177]) 
by spunkymail-mx8.g.dreamhost.com (Postfix) with ESMTP id BF7E14CF56 
for <me@mydomain.com>; Mon, 18 Jun 2007 13:54:13 -0700 (PDT) 
Received: from localhost (webmail3.y.sd.dreamhost.com [10.3.36.33]) 
by enforcer.dreamhost.com (Postfix) with ESMTP id A38B617D00A 
for <me@mydomain.com>; Mon, 18 Jun 2007 13:54:13 -0700 (PDT) 
X-Amavis-Sender: returns@chennells.com 
X-Quarantine-ID: <yvhvQEaDn52f> X-Spam-Flag: YES 
X-Spam-Score: 2.031 
X-Spam-Level: ** 
X-Spam-Status: Yes, score=2.031 tag=-999 tag2=999 kill=1
tests=[HTML_MESSAGE=0.001, HTML_TAG_EXIST_TBODY=0.221,
MIME_HTML_MOSTLY=1.703, UPPERCASE_25_50=0, URI_AFFILIATE=0.106]
Received: from godfather.dreamhost.com ([127.0.0.1]) 
by localhost (godfather.dreamhost.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yvhvQEaDn52f 
for <me@mydomain.com>; Wed, 13 Jun 2007 20:45:55 -0700 (PDT) 
Received: from ss21.chennells.com (unknown [69.41.174.14]) 
by godfather.dreamhost.com (Postfix) with ESMTP id D6C2294DA4 
for <me@mydomain.com>; Wed, 13 Jun 2007 20:45:54 -0700 (PDT) 
Received: by ss21.chennells.com (Postfix, from userid 0) id E9F4CEBF9FF; Thu, 14 Jun 2007 05:45:52 +0200 (SAST) 
To: me@mydomain.com 
Subject: The Wild Groove Yonder 
From: "Groovetickets" <newsletter@groovetickets.com> 
Message-ID: <1181792752_SectionID-160396_HitID-1181773512000_SiteID-16491_EmailID-44109423_DB-2@ss21.chennells.com> 
Content-Type: multipart/mixed; boundary="86c019aa7d019ac51bb5d178260b3ee0" Mime-Version: 1.0 Date: Thu, 14 Jun 2007 05:45:52 +0200 (SAST)

I won't analyze all the lines, but note the ones that are important:

NOTE ON WHITELISTING

If messages from senders you whitelisted are getting quarantined, you aren't whitelisting the actual sender. While the message is still in the Junk Mail folder, click on its subject to show the message. On top, you should see the 'From' address, and next to it another address in ( ). That second address is what you should add to your whitelist. The 'Send to Inbox and Whitelist' button may or may not pick the right address, make sure you double check after using that and manually add the address or domain if needed. This method only works if you're not using IMAP filtering. If you are, you could check the full headers or the apparently mis-handled messages and look for an 'envelope sender' which is most likely the real sender. Newsletters and discussion lists especially like to show a different from address, so even whitelisting the full domain may help.

• X-Original-To: me@mydomain.com -- this is the real recipient, regardless of what the 'To' field shows. The junk settings of this address will determine how the message is handled.

• X-Spam-Status: Yes, score=2.031 tag=-999 tag2=999 kill=1

(please note, these are from a message that was quarantined)

• score=2.031 -- whatever score Spam Assasin gave to the message (see 'tests' right below it with sample explanation for the scoring)
• tag=-999 -- the domain has junk filter settings, so it was handled as such
• tag2=999 -- this is the tag level you set for your address
• kill=1 -- this is your quarantine setting - in this case, the score is higher than the quarantine number, so the message ended up in the Junk Mail folder. I moved it to the Inbox so I can analyze the headers.

Here is a score example when the message was sent directly to the Inbox:

X-Spam-Status: No, score=-2.171 tagged_above=-999 required=999
tests=[BAYES_00=-2.312, HTML_IMAGE_RATIO_06=0.139, HTML_MESSAGE=0.001,
UNPARSEABLE_RELAY=0.001]

The scores are named differently, but it's still pretty straignforward: score is the junk score and it's based on the tests tagged_above just shows that the domain is filtered required is the address' tag setting no kill is displayed

Here is a score example of a message from a whitelisted sender, also sent to the Inbox, without actual scoring:

X-Spam-Status: No, score=x tagged_above=-999 required=999 WHITELISTED tests=[]

So, all that above should give you a basic understanding how Spam Assassin filtering works. This is not set to stone, and SA can slip, landing whitelisted senders in the Junk Mail folder, scoring X when there should be a number (this happens when the message itself is above a certain size, because the filter is set to ignore those), etc. We apologize for that, and are currently looking into a better filtering solution. Don't hesitate to contact support if you have questions about headers and apparently misbehaving junk filters.

How Junk Mail works on primary addresses and forwards

OK, I admit, this can get a bit confusing. I'll try to be as graphical and straightforward as possible, but if you still have questions, don't hesitate to submit a support request, and we'll be happy to further explain.

Primary email address: Fairly simple - if the domain is being filtered, the email will be delivered based on the junk settings for that primary address. Reminder: make sure your domain is either pointed to our name servers or you have updated your name server provider for that domain with the new filter MX records.

Forwarding email address

• If the domain is being filtered, and the address is being forwarded to another DreamHost email address which is also filtered: email gets sorted based on the junk mail settings of the receiving mailbox/primary address. You don't need to set separate settings for the forwarding addresses, as they will just inherit those.
• If the domain is being filtered, and the address is being forwarded to another DreamHost email that is not filtered or is being forwarded to an outside address only (like @gmail), then the settings you created in webmail for that email address will not apply, and the tag and quarantine numbers will reset to 999/999. It can be confusing, as you may even see a junk folder when you log in (if the forward is to another DH email address), but the mail coming to this forwarding address will not get filtered. The reason for that is that you would be unable to check any quarantined messages. Please note: even if you are forwarding the mail to a mailbox that is filtered, if you have at least one outside forward, that will prevail and your settings will go back to 999/999.
• If the domain is NOT being filtered and it forwards to a DH email address that is on a filtered domain, the mail will not get filtered. The way our junk filter works, it checks the email addresses against the mail database on our filtering mail machines. If the address is not there, it sends it to a regular mail machine. Once the mail enters our system, it will not go out again to be filtered: once the mail hits our system through the non-filtered email address, it will not get filtered, even if the end of the forward is filtered address. Makes sense?

Removing Junk Filter from your domain

You can turn off the junk filter where you turned it on in the control panel: Anti-spam Please note, the DNS propagation rules apply again; it can take up to 4 hrs. Again, if you have outside name servers, don't forget to update them with the new MX records. You'll find the new records by clicking on the DNS link next to your domain name at Manage Domains .

Once the junk filter is off your domain (and propagation has finished), your mail will no longer get filtered and you won't have access to the messages you left in the junk folder (unless you used the IMAP filtering method). Do not turn on and off the filters if you are trying to fix something, let us know first. By turning off the filters, you lose all settings, and when you turn it back on, you will have to start from the default.

Using keyword filters with Junk Filter

You can still use the keyword filters even if the domain is set up with the junk filters. Mail will come in through the filter machines first, then gets routed to the regular mail machines where your keyword filters will apply.

Accessing Junk Mail folder from alternative webmail and other email clients

For now, the 'Junk Mail' folder is inaccessible from any webmail app other than DreamHost's copy of SquirrelMail. See this thread on the forums. If you wish to be able to access your junk mail through other clients, please make your voice heard: cast your vote on our Suggestions page

Note, if you enable IMAP filtering, you will be able to check your filtered mail on any IMAP client, including other webmail clients. You still won't be able to access the Junk Folder, but you only need that for blacklisted messages.

A Note on Attachments

The program we use for junk filtering has a very aggressive virus filter. This runs before it will even gets to the point where it would compare the scores to your settings or your blacklist/whitelist. Most attachments are suspect, and incorrectly coded headers can fall 'victim' of the virus scan. If you need to regularly receive attachments or even files inside the message body (considered attachment) on your address, you want to just pass on junk filtering. Again, raising your quarantine limit or whitelisting the sender won't do any good: the virus filtering happens before the rest of the evaluation. Turning off junk filtering on your domain is not the end of junk filtering: you could just apply message filters in the panel under Mail / Message filters. We're working on getting it enabled on a per domain basis.

Bayes

We use Bayes rules for filtering, and download them daily to keep 'em fresh from SpamAssassin and Openprotect.
We aren't using Bayes db yet, which is generated by the user based on 'learning' with ham/spam.
Our SpamAssassin does have an option to learn on its own, so that learning and the daily updated rules
create the Bayes score in your filtered email.

NOTE: The Bayes score can be negative, and it's like that to offset the other potentially false negative score that's based on the other filters.

Using your own SpamAssassin install

You may wish to download your own version or install ours into your home directory, so you can set up your own custom rules or train and create Bayesian database. Please note, we don't support custom procmail configurations, so you can only go with this setup if you have a private server with courier enabled. That way, you can access your filtered mail directly on the server.

Troubleshooting

Here are some basic tips on what to look out for if the filter doesn't work as expected. Most of this information is already in the text above, but I figured it wouldn't hurt collecting them in one place :)

• No messages are getting filtered at all

- make sure you turned on the filters in the panel - you have waited long enough for the new MX records to propagate (about 4hrs) after turning on the filter - if your DNS is not pointed to us, you made sure to update the other provider with the correct MX records (found in control panel under DNS at https://panel.dreamhost.com/index.cgi?tree=domain.manage&

• You can't see the Junk Folder when you log into webmail or the Spam Assassin configuration under Options

- make sure junk filter is turned on for your domain - make sure you log in with your email address, not with your mailbox name

• Filter is working, but you need it to be more restrictive

- edit the default filter settings in webmail under Options/Spam Assassin Configuration or through the panel under Mail/Anti-spam

• Whitelisted addresses still getting delivered to Junk Folder

- Be sure to read that note above about whitelisting for more information

• Junk Mail folder is still listed and you're still getting Junk Reports even though you have turned off the filter a while ago

- make sure you did turn off the filtering for that domain

• Filtering is on, but no messages are coming to the Junk Mail folder listed on Webmail

- make sure you're not using IMAP filtering - if you just recently changed, it may have stuck, so notify support - make sure your filtering is restrictive enough to catch junk mail