Client/server message-based architectures over the internet

The Problem

I've been thinking for some time now about the intricacies of message-based architectures when one of the channels for messages to pass along includes the internet.

I'm fine with the concept of queuing messages up on clients that, when processed, result in web-service calls over the internet that then queue those same messages on the server to be processed against business rules. What I don't currently know of is a mechanism of sending messages back to the client to tell it that something happened.

The Example

Consider for this example a desktop warehouse management application that wishes to receive messages from a server, rather than polling to see if there are any.

E.g.

1. User creates urgent order on website with tight SLA for picking within a few minutes, because someone is already on their way to collect the item.
2. OrderCreated message is published on the server.
3. Magic happens to send the OrderCreated message to the client software running on a PC in a warehouse.
4. The client software displays the new order on the screen (and perhaps caches to disk in case of restarts etc).
5. Warehouse team pick the items to fulfill the order.

It's point 3 above that I'm looking for technologies to fill the gap.

Some Possible Solutions

1. Polling the server regularly and often (sub-minute) to see if there are messages, and download them to a local queue for processing.
2. Use something like two-way comms in WCF, though this requires channels/sockets/ports etc are kept permanently open between clients and server, so it won't scale elegantly.
3. Actually open up ports and configure port forwarding and firewalls etc in local sites to enable MSMQ (or some other tech like WCF, which could mean hosting IIS or similar on clie开发者_如何学JAVAnt hardware) to push messages to clients. Of course the deployment and support costs here are astronomical.

Does anyone have any experience of achieving this with a solution I've not listed here? I'm not looking for details necessarily, I'm just poking around in this space to see how it might be done.

Many thanks in advance.

I have a lot of experience in the automation field, and I'd say that polling and two-way comms are the most common.

Of the two, I've always preferred two-way comms. It does scale pretty well up to a certain point - tens of thousands of socket/WCF connections are possible as long as the server uses asynchronous APIs (and in particular does not use the "one thread per connection" design abomination).

The polling method does create more unnecessary network traffic, but it may be better if you're scaling past a single server. It seems to me that you can scale out to a web farm fairly easily using existing technology, and such scaling (I think) would be easier if you're using a polling method. I've never had to scale that far, though.

I haven't used MSMQ for this type of communication, but have used IBM's WebSphere (a similar technology). Message queueing comes with its own set of problems for automation, such as handling dead-letter queues, etc. It's useful if you expect the server to be down for a period of time (or if you have unreliable "middleware" between client and server), but in general I've preferred bidirectional communication over sockets or WCF.

Perhaps pattern something after JMS (or even just build a JMS adapter for your existing .net server and then use JMS outright)? As in, you could have the client maintain a persistent socket connection to your notification server (if you're considering polling the server several times a minute, then why not just keep a socket open?). When the socket is established the connecting client can subscribe/register for whatever events it is interested in receiving (e.g. "I only want events that are associated with products in Warehouse A"). Then when the OrderCreated message is published on the server, the server can look through its client list to see who is registered for the message, and immediately post a notification to the client over the corresponding socket connection.

Here is a short code snippet that I previously used (3 or 4 years ago) to create an instant messaging program. It creates a TcpListener that can accept messages from the server, and a NetworkStream and TcpClient to send messages to the server. It may not be a complete answer, but it may help direct you.

I would use a WCF service using TCP and callbacks.

What makes you say that it doesn't scale?

I'm currently working on something very similar, where messages need to be delivered in timely fashion across the Internet to client software behind a DSL router/modem, and possibly bound to a dynamic IP address. Frequent polling is our chosen solution, striking a balance between performance and simplicity.

Take a look at WebSockets - it's not there yet, but it's intended to be the successor to Comet/long-poll techniques. It's intended for web browsers to implement, but no reason why you can't use in in another type of client.