Rails Rake task displayed execution expired message and program stopped

I have a rake task to load car's image from the table websites using paperclip. The image stored in database as a remote link.

Here is my code and i'm using ruby 1.8.7, rails 2.3.8 and DB mysql.

namespace :db do

  task :load_photo  => :environment do
  require 'rubygems'
  require 'open-uri'
  require 'net/http'
  require 'paperclip'
  begin
  images =Website.find(:all,:conditions=>["image_url is not null"])
  images.each do |photo|
     url = URI.parse(photo.image_url)
     Net::HTTP.start(url.host, url.port) do |http|
         if http.head(url.reque开发者_如何学Gost_uri).code == "200"
           Car.update_attribute(:photo,open(url))
         end
     end
  end
  rescue Exception => e
  end
 end 
 end 

Run above rake task by db:load_photo. In my table (Website) has 60,000 rows. Rake task running upto 10000 rows only and execution terminated with an error message "execution expired".

Can any one help me to figure this out?

Thanks in advance.

You may find it more performant to run it in batches, active record has a find_in_batches method which stops loading all the records into memory at one time.

http://ryandaigle.com/articles/2009/2/23/what-s-new-in-edge-rails-batched-find

You could change your code to look like:

namespace :db do
  task :load_photo  => :environment do
    require 'rubygems'
    require 'open-uri'
    require 'net/http'
    require 'paperclip'
    Website.find_in_batches(:conditions=>["image_url is not null"]) do |websites|
      websites.each do |website|
        begin
          url = URI.parse(website.image_url)
          Net::HTTP.start(url.host, url.port) do |http|
            if http.head(url.request_uri).code == "200"
              Car.update_attribute(:photo,open(url))
            end
          end
        rescue Exception => e
        end
      end
    end
  end 
end

I can only guess, but it looks like you're making a little DoS attack to the server you're pulling images from.

You can try to play with a little delay between sequential requests (like "sleep 1").

Also, if your "execution expired" is a Timeout::Error exception, then you can't catch it with

rescue Exception => e

because Timeout::Error is not a subclass of StandardError, it is a subclass of the Interrupt class. You have to catch it explicitly, like so:

rescue Timeout::Error => e