python twisted threading
Hi can you please tell me how use different functions in different thread using thread pool in twisted...say
I have a list of ids x=[1,2,3,4]
where 1,2,...etc are ids(I got from data base and each one contains python script in some where disk).
what I want to do is
scanning of x traverse on list and run every script in different thread until they completed
Thanx Calderone, your code helped me a lot.
I have few doubts like I can resize threadpool size by this way.
from twisted.internet import reactor
reactor.suggestThreadPoolSize(30)
say all 30 available threads are busy & there is still some ids in list(dict or tuple) 1-In this situation all开发者_开发技巧 ids will be traversed? I mean as soon as thread is free next tool(id) will be assigned to freed thread? 2-there is also some cases one tools must be executed before second tool & one tool output will be used by another tool,how will it be managed in twisted thread. 3
Threads in Twisted are primarily used via twisted.internet.threads.deferToThread
. Alternatively, there's a new interface which is slightly more flexible, twisted.internet.threads.deferToThreadPool
. Either way, the answer is roughly the same, though. Iterate over your data and use one of these functions to dispatch it to a thread. You get back a Deferred
from either which will tell you what the result is, when it is available.
from twisted.internet.threads import deferToThread
from twisted.internet.defer import gatherResults
from twisted.internet import reactor
def double(n):
return n * 2
data = [1, 2, 3, 4]
results = []
for datum in data:
results.append(deferToThread(double, datum))
d = gatherResults(results)
def displayResults(results):
print 'Doubled data:', results
d.addCallback(displayResults)
d.addCallback(lambda ignored: reactor.stop())
reactor.run()
You can read more about threading in Twisted in the threading howto.
精彩评论