Page caching using delayed job

Hey all, if you've ever posted on [craigslist], this question should make sense to you. Whenever you post a listing (to sell furniture or an apartment, for example), your listing is not immediately thrown up on the site. Rather, listings will appear in batches (numbers vary) about every 10-15 minutes. At first I was开发者_C百科 really over-thinking this behavior, trying to hold records and then do mass inserts, but I realized it was much simpler. After talking with some colleagues, it made sense that Craigslist is caching their pages and then emptying that cache every 10-15 minutes. This severely decreases the load on their database.

Now, to my question. How do I accomplish the same thing in Rails? I know how to implement caching - I've read the [caching with Rails guide]. I will be using action caching and fragment caching (because I can't cache the whole page). I still need to do validations and access controls, so I can't fully cache the page...

To accomplish timed page caching you can utilize the standard Rails' caching plus a little timed cleverness.

First you want to determine your level of caching. You've got three options:

1. Page Caching - caches the whole page but subsequent requests don't go to through Rails stack. So if this is a Craiglist-esque page that will be hit thousands of times a second this request will only go to your webserver (e.g. apache) not Rails or your db making it much faster. The trade off is that you lose authentication, session variables, etc that Rails provides.
2. Action Caching - caches the whole page but brings the request into Rails so that it can execute any filters associated with that action.
3. Fragment Caching - caches a segment of the page, essentially bypassing the need to execute the code with in the block (and any consequential calls to the DB).

Then you'll need pick the appropriate level of caching and implement it in your app (check out the links above for implementation examples).

Once you have implemented the caching you now have to figure out a way to expire the cache. I can think of two ways to do this, both come with benefits and drawbacks. For now let's assume you've chosen to use action caching.

1. Reliable but more involved - create an action within your controller that expires the cache and a cron job task that makes a request to that action. I've asked a similar question that addresses this 'built in' scheduled task. For security precautions, you may want to include a generated hash or something similar so that someone can't manually expire your cache by going to '/products/expire_cache'.

   class ProductsController < ApplicationController
     caches_action :index
   
     def index
       # implementation
     end
   
     def expire_cache
       if params[:verification_hash] == 'sa89sf8sfsfehiwaf89yfea98fh'
         expire_action :action => :index
       end
     end
   end
   

2. Unreliable but easier - simply expire the cache in your action with an arbitrary conditional. This implementation assumes that there will be enough traffic to regularly ensure that someone will come to your site on the 0, 15, 30, and 45 minutes. You could decrease this interval in order ensure that the cache will be reset at a more probable interval.

   class ProductsController < ApplicationController
     caches_action :index
   
     def index
       # implementation
       expire_action :action => :index if Time.now.min % 15 == 0
     end
   end
   

I think there are two approaches to what you want to accomplish:

1. Simply cache with fragments and actions so that on the first hit of the page the database is accessed and the page loads normally, but every subsequent hit is from the cached version. The major upside of this approach is that you don't need to deal with delayed jobs and rendering your pages outside of the regular flow of things.
2. Create a delayed job that renders your page or the individual fragments that get cached. During the rendering in the delayed job the page fragments will actually get cached as if a user were viewing them (provided you have implemented fragment and action caching normally). Once the delayed job is done, populate a column in your database that indicates that this record/page is ready for viewing.

Probably the easiest way for caching in Rails is using Memcached with :expires_in option.

You will probably need a VPS server to use it which could be expensive for smaller sites.

But you can do good time based caching even without memcached. This little SimpleCache snippet has worked wonders for my shared hosted sites.