w3wp occasionally hangs on nightly recycle

I have come across an issue with our ASP.NET application where occasionally a nightly recycle cases the w3wp to hang.

This is what happens:

Recycle is triggered. obviously this forces ThreadAbortException on all running threads. However, it doesn't seem to trigger a new w3wp, or it is the new w3wp that actually throws the exception (haven't been able to reproduce it yet).

In my logs I get a lot of ThreadAbortException, and the thread count just goes up and up indefinitely, meaning that any new request spawns a new thread that never finishes. If this would have been the old 开发者_Go百科w3wp, any new request would be routed to the newly started w3wp.

Neither the shutdown timeout nor the rapid-fail protection seems to trigger as well, leaving the saite unavailable until it is manually recycled. Most of the time it also eats up a lot of CPU leaving the server almost unusable as well.

We are using Monorail MVC which probably doesn't have anything to do with it, however we do utilize their RescueController system. If we we're to unintentionally catch the ThreadAbortException in our error handling, could that cause a infinite loop that would hang the w3wp so badly that IIS cant recover from it?

IIS is unable to start up a new worker process successfully due to a resource constraint, likely due to not enough free memory.

Try adjusting your Private Memory Limit in IIS, decreasing the number of Maximum Worker Processes aka Web Garden (rarely set it to greater than 1--if it's set higher than that there is likely an underlying problem to solve), increasing the physical RAM available on your servers, calling Marshal.ReleaseComObject appropriately, and otherwise sorting out what is preventing freeing of memory. You might even consider changing the garbage collection mode from Server to Workstation (see http://msdn.microsoft.com/en-us/library/ms229357.aspx).

It turns out that we had an recursive try catch loop for Exception, that catches ThreadAbortException and invokes itself because of inheritance, so it became an infinite recursion.

We had the catch for Exception because we wanted logging and some error handling, and would probably be fine in all other aspects except that ThreadAbortException will keep on throwing during the execution.