Threads in Python again

guys!

My application is a bot. It simply receives a message, process it and returns result.

But there are a lot of messages and I'm creating s开发者_开发百科eparate thread for processing each, but it makes an application slower (not a bit).

So, Is it any way to reduce CPU usage by replacing threads with something else?

You probably want processes rather than threads. Spawn processes at startup, and use Pipes to talk to them.

http://docs.python.org/dev/library/multiprocessing.html

Threads and processes have the same speed. Your problem is not which one you use, but how many you use.

The answer is to only have a fixed couple of threads or processes. Say 10. You then create a Queue (use the Queue module) to store all messages from your robot. The 10 threads will constantly be working, and everytime they finish, they wait for a new message in the Queue.

This saves you from the overhead of creating and destroying threads. See http://docs.python.org/library/queue.html for more info.

def worker():
    while True:
        item = q.get()
        do_work(item)
        q.task_done()

q = Queue()
for i in range(num_worker_threads):
     t = Thread(target=worker)
     t.daemon = True
     t.start()

for item in source():
    q.put(item)

q.join()       # block until all tasks are done

You could try creating only a limited amount of workers and distribute work between them. Python's multiprocessing.Pool would be the thing to use.

You might not even need threads. If your server can handle each request quickly, you can just make it all single-threaded using something like Twisted.