Custom Mail CNAME
When setting up custom a MX record you will usually want to also setup an A record for the mail sub-domain (mail.domain.com), which points to your mail server. This is doable using the control panel, but if you need to add a CNAME record for the mail sub-domain, you will run into a problem.
DreamHost's system automatically generates MX records for your mail sub-domain, which often is fine, since these records can coexist with an A record. But a CNAME is an alias to another machine which takes on its MX records as well, so a CNAME can't exist with other types of records. This will make the panel give you an error if you try to add a CNAME for "mail" (usually for GMail/Google Apps) to point to ghs.google.com)
Basically, there is only one solution. You must contact support and ask them to place a special parameter on your account that disables the automatic generation of these MX records so you can add your CNAME. DNS must then be flushed (which should be done when the special parameter is added to your account). This will remove all MX records for mail.yourdomain.com (the other ones for yourdomain.com stay) and then you can add your CNAME record.
Here is what you can ask when contacting support:
I need to add a CNAME for mail.<mydomain>.com but I am unable to because MX records are automatically generated by your system for the mail sub-domain even when I use custom MX records. I have heard you can add a special parameter to remove these MX Records from my domain so I would ask that you please do this for me. Thanks!
Just replace <mydomain> with your own domain name.
The End Result
Before you contact Dreamhost, the non-edtable section might look like this:
Afterwards, the MX records for 'mail' should be removed like this: