Communication between local JVMs

My ques开发者_开发知识库tion: What approach could/should I take to communicate between two or more JVM instances that are running locally?

Some description of the problem:

I am developing a system for a project that requires separate JVM instances to isolate certain tasks from each other entirely.

In it's running, the 'parent' JVM will create 'child' JVMs that it will expect to execute and then return results to it (in the format of relatively simple POJO classes, or perhaps structured XML data). These results should not be transferred using the SysErr/SysOut/SysIn pipes as the child may already use these as part of its running.

If a child JVM does not respond with results within a certain time, the parent JVM should be able to signal to the child to cease processing, or to kill the child process. Otherwise, the child JVM should exit normally at the end of completing its task.

Research so far:

I am aware there are a number of technologies that may be of use e.g....

• Using Java's RMI library
• Using sockets to transfer objects
• Using distribution libraries such as Cajo, Hessian

...but am interested in hearing what approaches others may consider before pursuing one of these options, or any others.

Thanks for any help or advice on this!

Edits:

Quantity of data to transfer- relatively small, it will mostly be just a handful of POJOs containing strings that will represent the result of the child executing. If any solution would be inefficient on larger amounts of information, this is unlikely to be a problem in my system. The amount being transferred should be pretty static and so this does not have to be scalable.

Latency of transfer- not a critical concern in this case, although if any 'polling' of results is needed this should be able to be fairly frequent without significant overheads, so I can maintain a responsive GUI on top of this at a later time (e.g. progress bar)

Not directly an answer to your question, but a suggestion of an alternative. Have you considered OSGI?

It lets you run java projects in complete isolation from each other, within the SAME jvm. The beauty of it is that communication between projects is very easy with services (see Core Specifications PDF page 123). This way there is not "serialization" of any sort being done as the data and calls are all in the same jvm.

Furthermore all your requirements of quality of service (response time etc...) go away - you only have to worry about whether the service is UP or DOWN at the time you want to use it. And for that you have a really nice specification that does that for you called Declarative Services (See Enterprise Spec PDF page 141)

Sorry for the off-topic answer, but I thought some other people might consider this as an alternative.

Update

To answer your question about security, I have never considered such a scenario. I don't believe there is a way to enforce "memory" usage within OSGI.

However there is a way of communicating outside of JVM between different OSGI runtimes. It is called Remote Services (see Enterprise Spec PDF, page 7). They also have nice discussion there of the factors to take into consideration when doing something like that (see 13.1 Fallacies).

Folks at Apache Felix (implementation of OSGI) I think have implementation of this with iPOJO, called Distributed Services with iPOJO (their wrapper to make using services easier). I've never used this - so ignore me if I am wrong.

I'd use KryoNet with local sockets since it specialises heavily in serialisation and is quite lightweight (you also get Remote Method Invocation! I'm using it right now), but disable the socket disconnection timeout.

RMI basically works on the principle that you have a remote type and that the remote type implements an interface. This interface is shared. On your local machine, you bind the interface via the RMI library to code 'injected' in-memory from the RMI library, the result being that you have something that satisfies the interface but is able to communicate with the remote object.

akka is another option, as well as other java actor frameworks, it provides communication and other goodies derived from the actor model.

If you can't use stdin/stdout, then i'd go with sockets. You need some sort of serialization layer on top of the sockets (as you would with stdin/stdout), and RMI is a very easy to use and pretty effective such layer.

If you used RMI and found the performance wasn't good enough, i'd switch to some more efficient serializer - there are plenty of options.

I wouldn't go anywhere near web services or XML. That seems like a complete waste of time, likely take more effort and deliver less performance than RMI.

Not many people seem to like RMI any longer.

Options:

1. Web Services. e.g. http://cxf.apache.org
2. JMX. Now, this is really a means of using RMI under the table, but it would work.
3. Other IPC protocols; you cited Hessian
4. Roll-your-own using sockets, or even shared memory. (Open a mapped file in the parent, open it again in the child. You'd still need something for synchronization.)

Examples of note are Apache ant (which forks all sorts of Jvms for one purpose or another), Apache maven, and the open source variant of the Tanukisoft daemonization kit.

Personally, I'm very facile with web services, so that's the hammer which which I tend to turn things into nails. A typical JAX-WS+JAX-B or JAX-RS+JAX-B service is very little code with CXF, and manages all the data serialization and deserialization for me.

It was mentioned above, but i wanted to expand a bit on the JMX suggestion. we actually are doing pretty much exactly what you are planning to do (from what i can glean from your various comments). we landed on using jmx for a variety of reasons, a few of which i'll mention here. for one thing, jmx is all about management, so in general it is a perfect fit for what you want to do (especially if you already plan on having jmx services for other management tasks). any effort you put into jmx interfaces will do double duty as apis you can call using java management tools like jvisualvm. this leads to my next point, which is the most relevant to what you want. the new Attach API in jdk 6 and above is very sweet. it enables you to dynamically discover and communicate with running jvms. this allows, for example, for your "controller" process to crash and restart and re-find all the existing worker processes. this is the makings of a very robust system. it was mentioned above that jmx basically rmi under the hood, however, unlike using rmi directly, you don't need to manage all the connection details (e.g. dealing with unique ports, discoverability, etc). the attach api is a bit of a hidden gem in the jdk, as it isn't very well documented. when i was poking into this stuff initially, i didn't know the name of the api, so figuring how the "magic" in jvisualvm and jconsole worked was very difficult. finally, i came across an article like this one, which shows how to actually use the attach api dynamically in your own program.

Although it's designed for potentially remote communication between JVMs, I think you'll find that Netty works extremely well between local JVM instances as well.

It's probably the most performant / robust / widely supported library of its type for Java.

A lot is discussed above. But be it sockets, rmi, jms - there is a lof of dirty work involved. I would ratter advice akka. It is a actor based model which communicate with each other using Messages.

The beauty is, the actors can be on same JVM or another (very little config) and akka takes care the rest for you. I haven't seen a more cleaner way than doing this :)

Try out jGroups if the data to be communicated is not huge.

How about http://code.google.com/p/protobuf/ It is lightweight.

As you mentioned you can obviously send the objects over the network but that is a costly thing not to mention start up a separate JVM.

Another approach if you just want to separate your different worlds inside one JVM is to load the classes with different classloaders. ClassA@CL1!=ClassA@CL2 if they are loaded by CL1 and CL2 as sibling classloaders.

To enable communications between classA@CL1 and classA@CL2 you could have three classloaders.

• CL1 that loads process1
• CL2 that loads process2 (same classes as in CL1)
• CL3 that loads communication classes (POJOs and Service).

Now you let CL3 be the parent classloader of CL1 and CL2.

In classes loaded by CL3 you can have a light-weight communication send/receive functionality (send(Pojo)/receive(Pojo)) the POJOs between classes in CL1 and classes in CL2.

In CL3 you expose a static service that enables implementations from CL1 and CL2 register to send and receive the POJOs.