Ruby on Rails migrations are very slow

I have SQLite3 database, which is populated with some large set of data. I use migration for that.

3 tables will have following count of records: Table_1 will have about 10 records each record of Table_1 will be associated with ~100 records in Table_2 each record of Table_2 will be associated with ~2000 records in Table_3

Th开发者_JS百科e count of records will be about 10*100*2000 = 2000000

This takes a long time... Event, if i populate my database with about 20000 records, it takes about 10 minutes.

Also, i have noticed, that, during migration execution, ruby interpreter takes just 5% from CPU time and 95% remains unused ...

What the reason of such pure performance ?

Quite simply, inserting large amounts of records through manually saving AR objects one at a time is going to take years.

The best compromise between speed and "cleanness" (i.e. not a complete dodgy hack) for inserting large amounts of data is ar-extensions's (http://github.com/zdennis/ar-extensions) import method. It's not ideal, but it's better than any of the alternatives I could find, and the syntax is clean and doesn't require you to drop to raw sql (or anywhere close).

Example syntax:

items = Array.new
1.upto(200) do |n|
  items << Item.new :some_field => n
end
Item.import items, :validate => false

At least in mysql this will batch the records into a single INSERT statement with multiple sets of values. Pretty damn fast.

If you run each INSERT statement in it's own transaction, SQLite can be very, very slow. But if you run it all in one transaction (or a logical set of transactions), then it can be very fast.

Seed_fu could help, as discussed in this question