Rails 3 - cache web service call

In my application, in the homepa开发者_开发百科ge action, I call a specific web service that returns JSON.

parsed = JSON.parse(open("http://myservice").read)
@history = parsed['DATA']

This data will not change more than once per 60 seconds and does not change on a per-visitor basis, so i would like to, ideally, cache the @history variable itself (since the parsing will not result in a new result) and auto invalidate it if it is more than a minute old.

I'm unsure of the best way to do this. The default Rails caching methods all seem to be more oriented towards content that needs to be manually expired. I'm sure there is a quick and easy method to do this, I just don't know what it is!

You can use the built in Rails cache for this:

@history = Rails.cache.fetch('parsed_myservice_data', :expires_in => 1.minute) do
  JSON.parse connector.get_response("http://myservice")
end

One problem with this approach is when the rebuilding of the data to be cached takes quite a long time. If you get many client requests during this time, each of them will get a cache miss and call your block, resulting in lots of duplicated effort, not to mention slow response times.

EDIT: In Rails 3.x you can pass the option :race_condition_ttl to the fetch method to avoid this problem. Read more about it here.

A good solution to this in previous versions of Rails is to setup a background/cron job to be run at regular intervals that will fetch and parse the data and update the cache.

In your controller or model:

@history = Rails.cache.fetch('parsed_myservice_data') do
  JSON.parse connector.get_response("http://myservice")
end

In your background/cron job:

Rails.cache.write('parsed_myservice_data',
  JSON.parse connector.get_response("http://myservice"))

This way, your client requests will always get fresh cached data (except for the first request if the background/cron job hasn't been run yet.)

I don't know of an easy railsy way of doing this. You might want to look into using redis. Redis lets you set expiration times on the data you store in it. Depending on which redis gem you use it'd look something like this:

@history = $redis.get('history')

if not @history
  @history = JSON.parse(open("http://myservice").read)['DATA']
  $redis.set('history', @history)
  $redis.expire('history', 60)
end

Because there's only one redis service this will work for all your rails processes.

We had a similar requirement and we ended up using Squid as a forward proxy for all the webservice calls from the rails server. Squid was configured to have a cache-expiry time of 60 seconds.

http_connection_factory.rb:

class HttpConnectionFactory
  def self.connection
    AppConfig.use_forward_proxy ? Net::HTTP::Proxy(AppConfig.forward_proxy_host, AppConfig.forward_proxy_port) : Net::HTTP
  end
end

In your application's home page action, you can use the proxy instead of making the call directly.

connector = HttpConnectionFactory.connection 
parsed = JSON.parse(connector.get_response("http://myservice"))
@history = parsed['DATA']

We had second thoughts about using Redis or Memcache. But, we had several service calls and wanted to avoid all the hassles of generating keys and sweeping them at appropriate times.

So, in our case, the forward proxy took care of all those nitty gritties. Please refer to Squid Wiki for the configuration parameters necessary.