Threading vs Multiprocess vs Twisted in Python

Can I get help converting this code from Threading to Mutl开发者_开发知识库iprocess. Then can anyone help convert this code usinf twisted.

Would there be a gain from using twisted to upload db

within Python vs External tools.

import  os, pyodbc, sys, threading, Queue


class WorkerThread(threading.Thread):
    def __init__(self, queue):
        threading.Thread.__init__(self)
        self.queue = queue

    def run(self):
        while 1:
            try: # take a job from the queue
                type  = self.queue.get_nowait()

            except Queue.Empty:
                raise SystemExit

            try:
               cxn = pyodbc.connect('DSN=MySQL;PWD=MLML;Option=3') 
               csr = cxn.cursor()    
               # Inserts,update, CRUD

            except:
               # count = count +1
                print 'DB Error', type

if __name__ == '__main__':
    connections =  25

    sml = ('A', 'B', 'C','D',)
    # build a queue with tuples
    queue = Queue.Queue()

    for row in sml:
        if not row or row[0] == "#":
            continue
        queue.put(row) 

    threads = []
    for dummy in range(connections):
        t = WorkerThread(queue)
        t.start()
        threads.append(t)

    # wait for all threads to finish
    for thread in threads:
        thread.join()

    sys.stdout.flush()

#csr.close()
#cxn.close()
print 'Finish'  

Updated: JP mentioned the txpostgres and txmysql modules, which I wasn't aware of. These allow Twisted to access both databases asynchronously (without using a thread pool).

Note that if you decide to use Twisted's enterprise adbapi, it will end up using a thread pool to handle database connections, roughly the equivalent of your existing example. See the docs on using Twisted's enterprise database module.

You should be able to directly translate your thread/queue-based code to use the multiprocessing module, by replacing your threads with Process instances, and uing the multiprocessing Queue implementation. See the multiprocessing docs.