Occasional "ConnectionError: Cannot connect to the database" to mongo

We are currently testing a django based project that uses MongoEngine as the persistence layer. MongoEngine is based on pymongo and we're using version 1.6 and we are running a single instance setup of mongo.

What we have noticed is that occasionally, and for about 5 minutes, connections cannot be established to the mongo instance. Has anyone come across such 开发者_如何学Gobehavior? any tips on how to improve reliability?

We had an issue with AutoReconnect which sounds similar to what you are describing. I ended up monkeypatching pymongo in my <project>/__init__.py file:

from pymongo.cursor import Cursor                                                             
from pymongo.errors import AutoReconnect                                                      

from time import sleep                                                                        
import sys                                                                                    

AUTO_RECONNECT_ATTEMPTS = 10                                                                  
AUTO_RECONNECT_DELAY = 0.1                                                                    

def auto_reconnect(func):                                                                     
    """                                                                                       
    Function wrapper to automatically reconnect if AutoReconnect is raised.                   

    If still failing after AUTO_RECONNECT_ATTEMPTS, raise the exception after                 
    all. Technically this should be handled everytime a mongo query is                        
    executed so you can gracefully handle the failure appropriately, but this                 
    intermediary should handle 99% of cases and avoid having to put                           
    reconnection code all over the place.                                                     

    """                                                                                       
    def retry_function(*args, **kwargs):                                                      
        attempts = 0                                                                          
        while True:                                                                           
            try:                                                                              
                return func(*args, **kwargs)                                                  
            except AutoReconnect, e:                                                          
                attempts += 1                                                                 
                if attempts > AUTO_RECONNECT_ATTEMPTS:                                        
                    raise                                                                     
                sys.stderr.write(                                                             
                    '%s raised [%s] -- AutoReconnecting (#%d)...\n' % (                       
                        func.__name__, e, attempts))                                          
                sleep(AUTO_RECONNECT_DELAY)                                                   
    return retry_function                                                                     

# monkeypatch: wrap Cursor.__send_message (name-mangled)                                      
Cursor._Cursor__send_message = auto_reconnect(Cursor._Cursor__send_message)                   
# (may need to wrap some other methods also, we'll see...) 

This resolved the issue for us, but you might be describing something different?

Here is another solution that uses subclassing instead of monkey patching, and handles errors that might be raised while establishing the initial connection or while accessing a database. I just subclassed Connection/ReplicasetConnection, and handled raised AutoReconnect errors during instantiation and any method invocations. You can specify the number of retries and sleep time between retries in the constructor.

You can see the gist here: https://gist.github.com/2777345