Python multiprocessing/threading with shared variables that only will be read
Considering the code below. 开发者_运维知识库I would like to run 3 experiments at a time. The experiments are independent, the only thing they share is the Model object which they only read.
As there are seemingly no hard things in threading this out, how can I best do this in Python? I would like to use a pool or so to make sure that only three experiments run at a time. Shall I use multi-processing? If yes, how is the shortest and most concise?
#!/usr/bin/env python2.6
import time
class Model:
name = ""
def __init__(self,name):
self.name = name
class Experiment:
id = 0
model = None
done = False
def __init__(self,id,model):
self.id = id
self.model = model
def run(self):
for _ in range(0,60):
print "Hey %s from experiment %d" % (self.model.name, id)
time.sleep(1)
self.done = True
if __name__ == "__main__":
experiments = []
model = Model("statictistical model")
for i in range(0,5):
experiments.append(Experiment(i, model))
#How to run 3 experiments at the same time
Check the docs, specifically:
http://docs.python.org/library/multiprocessing.html#module-multiprocessing.pool
There really are a lot of examples there that should get you on your way. For instance, I could come up with:
#!/usr/bin/env python2.6
import time
import multiprocessing
class Model:
name = ""
def __init__(self,name):
self.name = name
def run_experiment(id, model):
print "Experiment %d is starting" % id
for _ in range(0,60):
print "Hey %s from experiment %d" % (model.name, id)
time.sleep(1)
print "Experiment %d is done" % id
return "Result for %d" % id
if __name__ == "__main__":
model = Model("statictistical model")
experiments = ((i, model) for i in range(0, 5))
pool = multiprocessing.Pool(3)
results = [pool.apply_async(run_experiment, experiment) for experiment in experiments]
for result in results:
r = result.get()
# do something with r
# or nothing, i suppose...
Do also pay attention to what the docs say about using the multiprocessing
module:
Functionality within this package requires that the
__main__
method be importable by the children. This is covered in Programming guidelines however it is worth pointing out here. This means that some examples, such as themultiprocessing.Pool
examples will not work in the interactive interpreter
You must always keep in mind that threads do not really run parallel, if that is what you actually want.
I do not unterstand what the actual problem is...? If you only want 3 threads running the same time, why not just start only 3 threads?
精彩评论