Using WordPress with W3TC and DreamObjects

Warning: this page was edited by a happy DreamHost customer. I'm not a DreamHost representative and you should not take these guidelines as an official guide, or any official endorsement by Dreamhost. Caveat utilitor. Gwyneth Llewelyn (talk) 18:03, 19 October 2012 (PDT)

Line numbers are current for W3TC version 0.9.2.5. Gwyneth Llewelyn (talk) 09:33, 2 January 2013 (PST)

Be careful! These instructions require you to manually change a plugin, by hacking its code. If you do a mistake, it will mean that you might break the plugin, and, as a consequence, your site might stop working. Do NOT do this to a website which is important for you, or that cannot have any downtime. The preparation of the whole procedure takes a little time, depending on how complex your website is, and it can well mean an hour of downtime or more. And that is if you do not make any mistakes!

Preparing
Taking all that into account, to be sure, make a full backup of your WordPress installation. If you don't know how to do it manually, at least use an automatic backup tool. I personally recommend BackupWordPress which is free, simple to use, and does the job well.

The next step is to make sure you have W3 Edge's W3 Total Cache (W3TC) plugin installed. Please note that the last time I've checked, this particular caching plugin is not recommended by DreamHost. Why? Mostly for two reasons: it is very, very hard to configure properly; and it tends to consume a lot of memory, which will mean your WP processes will be killed by DH's watchdog to keep resource usage down.

If you are already familiar with W3TC, and have it installed, you can skip the next section.

Configuring WP for using W3TC
I have posted those instructions elsewhere, but you can follow this forum thread for some of the main configurations. Simply put: don't use PHP 5.3 FastCGI (it consumes more memory for increased speed), stick to PHP 5.3 CGI instead. Don't use Extra Web Security — register with CloudFlare, they provide the same functionality (with tons of extra functionality besides that, of course). Previously, Page Speed Optimization would also be a no-no (it would consume more memory to optimize the pages), but since DH switched to Google's Apache plugin, this seems to work fine.

So the idea is: on a shared DH server, memory is at a premium. With these settings, you're trading off raw speed and performance for less memory, to keep DH's shared servers happy. The point here is that most static content will be served by DreamObjects, and, if you wish, the rest can be distributed with CloudFlare — so you're not going to lose any performance that way.

Register for the DreamObjects beta and configure it
This is the easy part! Register for the DreamObjects beta and, from the DH Control Panel, set up a bucket on DreamHost and mark it as "public".

Hack the W3TC plugin code
Now comes the dangerous part!

Go to /wp-content/plugins/w3-total-cache/lib and edit S3.php on lines 819 and 1346, changing the hard-coded domain name s3.amazonaws.com to objects.dreamhost.com.

Then go to /wp-content/plugins/w3-total-cache/lib/W3/Cdn/S3.php and do the same on line 339, and, for good measure, change it on /wp-content/plugins/w3-total-cache/lib/W3/Cdn/S3/Cf.php on line 95 as well. This should be enough.

To make it look nicer on the Dashboard page for the W3TC plugin, change it as well on /wp-content/plugins/w3-total-cache/inc/options/cdn/s3.php, lines 36 and 38.

I used a text editor for making the changes, since not all of those files are available from the WP built-in plugin editor.

Configure W3TC to use CDN
On the WordPress Dashboard for your site, go to the General tab for the W3TC plugin (shown on the sidebar as "Performance"). I selected CDN and as a CDN Type Amazon Simple Storage Service. W3TC, at this point, complains that you need to fill in all the remaining data and push content to the cloud service; keep that message open.

Then go to the CDN tab and put your API key, secret, and bucket name. Test it (clicking the button below). It should take a second to give you a green status.

Move your initial (static) content to the CDN
If you haven't hidden the message to push all content over to the CDN, this is now the time to click on all those buttons. Sadly, you have to do them one at a time. If you have a lot of content — lots of images and so forth — this can take quite a while. Just be patient and go through all those buttons.

That's it! You're now serving all your static content from DreamObjects :-) Drool at the speed performance increase!

Even more speed: there is no problem having CloudFlare on top of everything. I did precisely that. My own website (a relatively low-traffic blog with just 5-7 GBytes of monthly traffic) used to take up to a minute for the first page to load. After DH moved everything to a new server (when the old one died), and with all the above improvements, now Pingdom tells me that the first page tends to load in 2.78s (Dallas) or 4.38s (Amsterdam) — that's for 220 requests and 1.3 MB of downloads!

Serving cloud content from your own subdomain
W3TC adds a nice feature, which is the ability to use your own subdomain for the content stored on the cloud. This means that the URLs from static content stored at DreamObjects could be served as yourbucket.yourwebsitedomain.tld, pointing to yourbucket.objects.dreamhost.com. That's how it works under Amazon S3.

Sadly, at the time of writing, DreamObjects doesn't allow that. But thanks to justinlund's explanation, there is another workaround that could be applied instead: point a CNAME for something.yourwebsitedomain.tld to objects.dreamhost.com, and rewrite the URL to go to something.yourwebsitedomain.tld/yourbucket. This works!

This requires a bit more hacking of the W3TC code, and

Go to /wp-content/plugins/w3-total-cache/lib/W3/Cdn/S3.php and scroll down to line 337.

Replace

with

instead (this has some caveats that I will address later).

Then go to /wp-content/plugins/w3-total-cache/lib/W3/Cdn/S3/Cf.php on line 95. If you haven't done the steps on the previous section, it should still say

(You might have changed that to

)

But we're now replacing that single line to:

Starting with W3TC 0.9.2.5, you have to do the same on line 232 as well.

Then go to the DreamHost Control Panel, select Manage Domains, and on the root domain name to which you want to add this feature, click on DNS. Go to where it says Add a custom DNS record to yourwebsitedomain.tld and add a new entry for something.yourwebsitedomain.tld, type CNAME, value objects.dreamhost.com and wait a few minutes until DreamHost has changed the DNS for your domain. If you use CloudFlare (I do!) it's time to do the same there.

Last step: go to your WP Dashboard, Performance > CDN, and scroll down to where it says Replace site's hostname with:. Add just one CNAME for something.yourwebsitedomain.tld.

Now save everything, clear the whole cache, and reimport whatever needs to be reimported (probably everything) to DreamObjects, by clicking on all buttons :) If you're using CloudFlare, for good measure, clear the cache there as well.

Why bother?
Some ranking algorithms, like Google's, don't like to see content being served from a subdomain that is not the same as the main domain. Why, I have no idea, but this should improve ranking with those algorithms.

More interesting is the combination with CloudFlare. After testing that everything works as expected, you can now CloudFlare-enable the something.yourwebsitedomain.tld subdomain. What happens now is that every static content stored at DreamObjects will now be cached by CloudFlare as well! Why is that important? Well, CloudFlare has 14 data centers across the world to serve the content from, so they will reach a wider audience with faster ping times. This is obviously more important if your audience is not all in the US, but even US viewers should see an even bigger improvement.

Caveats
The dirty hack assumes that there is only one single CNAME. W3TC, however, allows multiple CNAMEs — so that you can separate buckets for JS, for CSS, for images, and so forth. Well, I was looking just for a proof-of-concept, so I'm just returning the first CNAME and append the bucket name to it. If you really, really need separate CNAMEs and separate buckets (imagine sharing buckets with similar JS, images, etc. across several WP installs — it might make sense!), you will need to hack the code further.

Of course, if and when DreamHost changes CNAME support for DreamObjects to work like it does on Amazon S3, this will become irrelevant, and you can revert the code to what it was on the previous section. This is just a temporary workaround until the DH developers do their magic. They say they're working on it!

Caveats
The most obvious one is that, until W3 Edge adds an option for the WP admin to input the S3 API endpoint server, you will need to hack the code every time a new release of W3TC comes out. It's a bummer.

Also, remember that DreamObjects is still in beta. In my case, I experienced some connection errors when pushing and retrieving content from DreamObjects. If you start seeing red [503] Unexpected HTTP status messages when pushing content from W3TC to DreamObjects, you might have lost the connection to DreamObjects. A way to check that is to go to the DH Control Panel, click on the link for DreamObjects, and see if you can view the content inside your bucket. If you're getting the dreaded "Network Error. Try Again Later" message, it means that DreamObjects is unresponsive. During my tests this happened quite often, but after a few minutes, the message would disappear. Fortunately, W3TC will be perfectly able to pick up what it was doing (so it won't start from scratch; it will skip over the already up-to-date files). Also, the operations through the S3 API seem to be atomic: I never caught any file corruption even when DreamObjects was down.

To-do: If W3 Edge is unwilling to add the option to manually edit the S3 API endpoint, I might do it myself :) It's not impossibly hard, but takes a few hours. Of course, there is no guarantee that my changes might actually be implemented by W3 Edge, even if I send the code back to them.