Shared messaging queue for multiple worker threads in IIS

I'm currently trying to interface our new intranet (ASP-MVC) with the web front end of our pager system. The pager system's front end is about 10 years old, poor interface and no documentation. To do this I have the form on the intranet post to the server, our server then sends an HTTP POST to the pager server that mimics what its own form sends.

While testing this for overloading (we sent about 10 messages almost concurrently) the pager server crashed, as the system is a black box and the only difference we noticed was the concurrent POSTs the least we can do is try to prevent this happening.

I want to have all messages go into a queue that sends at most once every 5-10 seconds but I'm not sure how to implement this, does anyone have any suggestions or links to some basic tutorials to get me started please?

Update

I didn't specify t开发者_JAVA技巧his earlier but the process should be synchronous from the point of view of the browser, ie webserver gets the http POST request from the browser, message is put into the queue, message is sent from the queue, webserver sends http response to the browser. This is not a high volume service and I expect simultaneous requests to be rare so response will be near instantaneous, it is more of a throttle to prevent potential issues with concurrent requests.

Assuming, you don't want to loose any request, your queue needs to be persistent. You can roll up your own implementation and puts the data into database or you can use MessageQueue as ready-made backing store. Your intranet site will be pushing (sending) messaged into database or MSMQ. You can write windows service or a scheduled console application to read from queue (using polling) and then post the data to your pager system.

Here are few articles to start with message queue: http://www.primaryobjects.com/CMS/Article77.aspx and http://msdn.microsoft.com/en-us/library/system.messaging.messagequeue.aspx

Sounds like a job for queued messaging! :)

Some options on Windows, in no particular order, are:

1. MSMQ
2. Service Broker
3. Custom Sql Queue
4. Third party queuing products

With all these implementations, you insert the pager message into a queue, and then use a worker process to pop the messages off the queue and send to the downstream service. Your worker will control the flow rate from the queue.

If the pager service goes down for whatever reason, you don't send the message - it gets left in the queue to be retried again.

Search the googles for "Getting Started with [Insert Technology Here]" and I'm sure you'll get enough information and sample code for you to get started.

MSMQ

MSMQ flat out rocks and is easy to work with. It has a couple of limitations out of the box such as 4MB message size and making queues highly available (each queue instance is typically backed by local storage on a server).

4MB message size can be overcome easily if you own both ends of the queue (both sender and receiver) as you can chunk your messages through the queue (break a message > 4MB into smaller chunks and reassemble on the receiver.

Overcoming the HA limitation typically takes assistance with storage and cluster resources - its probably easier to be able to recover messages from source system than fool with clustering MSMQ (this sounds wrong to me, but has been my experience - would love for MSMQ guru to blog about successful HA).

Service Broker

Service Broker is a great point to point queuing solution built into Sql Server (even Sql Express has it). Its only drawback is the lack of tooling which makes configuration management difficult.

Service Broker is a good choice if you have multiple queue consumers from a single queue...this makes for a HA queueing solution assuming Sql Server is HA.

Custom Sql Queue

This can be a good choice if your comfortable with controlling the batching of units of work out of your queue via T-SQL. Something that Service Broker will handle for you.

Custom SQL queue is a good choice if you have multiple queue consumers from a single queue...this makes for a HA queueing solution assuming Sql Server is HA.

Third Party Queueing Products

There are a number of 3rd party queueing products out there that can be inserted here - though they may not be Windows, or even .NET specific. This can be a good solution if your environment is looking to implement "queueing as a service". You can get .NET bindings for lots of good queuing implementations. Check out implementations of Advanced Message Queueing Protocol for more information.