开发者

JVM (embarrasingly) parallel processing libraries/tools

I am looking for something that will make it easy to run (correctly coded) embarrassi开发者_开发问答ngly parallel JVM code on a cluster (so that I can use Clojure + Incanter).

I have used Parallel Python in the past to do this. We have a new PBS cluster and our admin will soon set up IPython nodes that use PBS as the backend. Both of these systems make it almost a no-brainer to run certain types of code in a cluster.

I made the mistake of using Hadoop in the past (Hadoop is just not suited to the kind of data that I use) - the latency made even small runs execute for 1-2 minutes.

Is JPPF or Gridgain better for what I need? Does anyone here have any experience with either? Is there anything else you can recommend?


Check out cascalog - http://github.com/nathanmarz/cascalog


Clojure is reported to work on Terracotta, subject to some patching.


Look at Skandium

Edit :

Above link is no more live, so adding github link

https://github.com/mleyton/Skandium


I suggest you look at Skandium, alternative licenses to GPL can be negotiated with the developers upon request.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜