开发者

Does Python's multiprocessing.Pool support remote subprocesses? [duplicate]

This question already has answers here: Using the multiprocessing module for cluster computing (4 answers) Closed 10 years ago.

I have recently discovered how awesome Pool.map() is for simple multiprocessing tasks, but my local machine only has two processors and I was hoping to take advantage of some remote computing resources.

Can a processing pool be constructed to use remote machines as well as the local machines in a transparent way?

What python code would I need to run on the remote machines, How would I set up the pool on the local (master) machine?

On the documentation page for mul开发者_JS百科tiprocessing, It says Managers are to be used for shared objects, but it's unclear to me how they should be used for a remote Pool. Nothing other than Managers seems make any reference to ports and addresses, so I'm guessing all remote activity happens through this class.


multiprocessing.Pool seems not designed for distributed processing -- if that's what you want to do, I suggest you check out the execnet package instead. That works over SSH pipes and sends code along to the remote Python interpreter if necessary.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜