Data iterators in Rails?

When I do

users = User.find(:all)

in Rails, from my understanding, it pulls in all users from my database.

This seems like it could easily eat your server's memory if you have thousands of user records. Is this true? If it is true, is there a way to iterate over data in a table without killin开发者_开发问答g your server?

Since rails 2.3 you can use find_each and find_in_batches:

User.find_each { |user| user.some_method }

will load a default of 1000 users. You also have the :batch_size option to change the default size.

Find in batches is similar but it provides the array to the block instead of the single object:

 User.find_in_batches do |users|
  users.each { |user| user.some_method }
end

Yes, it's definitely true. There may be some "object pagination" plugins for that purpose, but I have never looked into it.

One thing I do know is that ORMs like ActiveRecord tend to be very wasteful by default. Let's say you only need your user's ids and usernames, but there are actually 12 columns in the users table. By default ActiveRecord grabs all 12 columns, putting unnecessary load on both your database and your server's memory. To eliminate that waste in Rails 3:

users = User.select('id, username').all

I think Rails 2 would look something like

users = User.find(:all, :select => 'id, username')

Maybe not exactly what you were looking for, but that's cut out quite a few bottlenecks of mine.

Indeed, your method can easily consume a lot of server resources.

As an alternative to finding objects in batches, you can use will_paginate helper.