How to exit a loop of Python Multiprocessing?
I have a multiprocessing code that search database records for a certain condition, when it reaches the end of the database the loop must stop, how can I do that? Here is the code:
import psycopg2
from multiprocessing import Pool
conn = psycopg2.connect(a database)
query=conn.cursor()
query.execute(some query)
def scanfile():
try: row=query.fetchone()
except: return False #here the loop must stop
#do something with row...
if __name__=='__main__':
pool=Pool()
while True:
pool.apply_async(scanfile)
pool.close()
pool.joi开发者_运维技巧n()
Move your query / cursor over as the iterable and let multiprocessing distribute the work for you.
import psycopg2
from multiprocessing import Pool
def scanfile(row):
#do something with row...
if __name__ == '__main__':
conn = psycopg2.connect(a database)
query = conn.cursor()
query.execute(some query)
pool = Pool()
pool.map_async(scanfile, query).wait()
pool.close()
pool.join()
精彩评论