开发者

How to schedule jobs in to client systems using python?

i have 4 systems , where in one i am using as a master and the rest 3 as slaves. I want to execute a set of functions in client systems and fetch back the result, to achieve this i had previously used Parallel Python , unfortunately as soon as job_server is created in it , it internally executes the given function using all systems. I want to individually assign particular function to be executed on an individual client machine , i have no idea on how to proceed with coding . Is there any framework in python, which allows user开发者_JS百科s to do that ?


There is the multiprocessing module: http://docs.python.org/library/multiprocessing.html You can find examples there that do exactly what you want (see Remote Managers).

As a side notice: I soon stumbled upon limitations in the multiprocessing module and now use PyRo as middleware. It's very simple and powerful but requires a bit more work to set up a basic framework.


There was a talk at a Pycon in the past few years about a tool that managed remote processes via ssh, which sounded really cool and I meant to check out. I've forgotten what the tool was, but you might find it by looking at the Pycon talks for the past few years.

The Python Parallel Processing wiki page has a big list of tools that might help you.


I don't quite understand what you have now and what you're trying to do. Pyro is easy to use, pretty fast and well documented for letting multiple python processes (on multiple systems) interact with each other. Take a look at that. You'll need to ensure that something is running on all of them so they're listening for commands, but that's not too hard.


In Pythomnic you can have multiple processes on multiple servers exchanging synchronous calls like this:

result = pmnc("other_process").module.method(...) # invokes remote method

So you can eiher have 3 distinctly named slaves and pick call target manually:

pmnc("slave_1").foo.bar() # calls bar() in foo.py on slave_1

or have slaves identically named and have the framework pick any available (if they are interchangeable of course)

pmnc("slave").foo.bar() # calls bar() in foo.py on any slave


check out RPyc

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜