Rails activerecord-import: How do I know how many records to insert in one batch?

I've just started using activerecord-import, and it is much faster than the one by one inserts I was doing before. I'm currently inserting in batches of 500 records, and wondered what other people have found to be a good number for the batches? I know there is no "one size fits all" answer. I'm just looking for some guidance by others that have experience with activerecord-import. I've looked at the documentation and Googled it, and haven't found anything.

Additional info fwiw: I'm using Postgres, and saving model objects, not columns and arrays 开发者_StackOverflow中文版with values.

Any help is appreciated.

activerecord-import will break the import into groups based on max-allowed-packet for your database adapter, so you shouldn't run into any problems. Optimization is another issue.

The only thing I would be concerned with is the memory allocation - if you've already allocated all the objects and are just feeding them in batches to an import call, I'd do them all at once.

If you're generating the objects dynamically for each batch (while reading from file, for example), you should do some testing and see what the memory usage looks like for various batch sizes.