Insert performance with Cassandra

sorry for my English in advance.

I am a beginner with Cassandr开发者_开发技巧a and his data model. I am trying to insert one million rows in a cassandra database in local on one node. Each row has 10 columns and I insert those only in one column family.

With one thread, that operation took around 3 min. But I would like do the same operation with 2 millions rows, and keeping a good time. Then I tried with 2 threads to insert 2 millions rows, expecting a similar result around 3-4min. bUT i gor a result like 7min...twice the first result. As I check on differents forums, multithreading is recommended to improve performance. That is why I am asking that question : is it useful to use multithreading to insert data in local node (client and server are in the same computer), in only one column family?

Some informations : - I use pycassa - I have separated commitlog repertory and data repertory on differents disks - I use batch insert for each thread - Consistency Level : ONE - Replicator factor : 1

It's possible you're hitting the python GIL but more likely you're doing something wrong.

For instance, putting 2M rows in a single batch would be Doing It Wrong.

Try running multiple clients in multiple processes, NOT threads.

Then experiment with different insert sizes.

1M inserts in 3 mins is about 5500 inserts/sec, which is pretty good for a single local client. On a multi-core machine you should be able to get several times this amount provided that you use multiple clients, probably inserting small batches of rows, or individual rows.

You might consider Redis. Its single-node throughput is supposed to be faster. It's different from Cassandra though, so whether or not it's an appropriate option would depend on your use case.

The time taken doubled because you inserted twice as much data. Is it possible that you are I/O bound?