C# - Throttling a MessageQueue

I have code that sends a message into a MessageQueue.

_queue.Send(new Message(myData));

This queue is on the local machine and the threads that Receive() from it are in the same process.

What happens if the messages are inserted faster than they are extracted? Will Send() block? Is there a way for me to know if the MessageQueue is full before sending into it more events? (I would prefer at this point just to log myData and not send the 开发者_JS百科event).

Thanks, Sela.

Short answer: Do the simple thing and don't limit on send.

Long answer: The message queue will only really get full when the disk it's allocated to save to is out of space - which is the same time that your logging will be out of space. The message queue is very good at holding data you're not ready to process. Don't throttle on send. If you're concerned about system management and disk space then you might prefer to rely on Window's excellent system monitoring facilities and disk space usage threshold alerts. You don't need to reinvent this for your application.

That is unless you're running the queue in memory only mode which may not be necessary. If you can't process the messages fast enough then you definitely have enough time to let the queue manager persist the messages to disk. You should only consider running the queue in memory only mode if you're going to scale to many consumer processes on many servers and the disk IO on the queue manager becomes the bottleneck. One process on the same machine is very far away from that scenario. Let the queue manager do what it does best. Don't optimise prematurely.

If you implement a specified quality of service like X messages per second and bill your customers more for processing a higher quality of service then throttle at the receiving end. I've done this successfully using a semaphore initialised with a resource limit equal to the number of messages to consume per second. Each consumer thread took a snapshot of the message start time, processed 1 message and then waited for the end of second before giving up the semaphore. That way the thread pool could grow to accomodate the quality of service if messages took more than 1 second to process but would not exceed the quality of service.

Good luck!

Designing a system so that messages are produced no faster than they are consumed is good and I agree with that. Nevertheless it might happen that a producer of messages would hit a wall because queue is overfilled especially when there are low quotas set.

To prepare for such a situation you need to monitor if Send() method succeeds. If you send a message into a full queue the message is lost and because Send() returns void there is no immediate result of success or failure. But there is a way to detect that. You should request an acknowledgements when working with MSMQ. To receive them you need to use an administrative queue. This way you can be notified about different things that happen including queue being full.

Message msg = new Message
{
    Formatter = new BinaryMessageFormatter(),
    Body = data,
    AdministrationQueue = this.adminQueue,
    AcknowledgeType = AcknowledgeTypes.FullReachQueue
};

this.queue.Send(msg);

Message admMsg = this.adminQueue.Receive();
if (admMsg != null && admMsg.Acknowledgment == Acknowledgment.QueueExceedMaximumSize)
{
    // queue is full
}