Twisted: degrade gracefully performance in case reactor is overloaded?

Is it somehow possible to "detect" that the reactor is开发者_JS百科 overloaded and start dropping connections, or refuse new connections? How can we avoid the reactor being completely overloaded and not being able to catch up?

If I understand Twisted Reactors correctly, they don't parallelize everything. Whatever operations have been queued is scheduled and is done one by one.

One way out for you is to have a custom addCallback which checks for how many callbacks have been registered already and drop if necessary.

No easy way, but here's some suggestions: http://www.mail-archive.com/twisted-python@twistedmatrix.com/msg00389.html

I would approach this per protocol. Throttle when the actual service requires it, not when you think it will. Rather than worrying about how many callbacks are waiting for a reactor tick, I'd worry about how long the HTTP requests (for example) are taking to complete. The number of operations waiting for the reactor could be an implementation detail - for example, if one access pattern ended up with callbacks on long DeferredLists, and another had a more linear chain of callbacks, the time to respond might not be different even though the number of callbacks would be.

This could be done by keeping metrics of the time to complete a logical operation (such as servicing a HTTP request). An advantage of this is that it gives you important information before a problem happens.