Gunicorn, Django, and slow loading code...?

My code uses matplotlib which takes a good second or two to load (it seems). The problem is that when I send kill -HUP <pid> to Gunicorn, that the next request to Gunicorn loads the actual code at request time which takes a good second or two.

I'd like to not force the user to wait that second or two. I've been playing around with hitting the website with urllib2.urlopen() and forcing some instances to load the code, but I cannot guarante开发者_运维问答e that all workers have loaded the code.

How can I work around the backend loading the code at request time vs when its started? Do I put the offending slow loading modules in settings.py?

Gunicorn has some pretty awesome configuration. Was browsing around a git clone of their source and found an example_config.py which has a "post_fork" which got me thinking. After some poking around the source a bit more, I came up with this solution.

% gunicorn_django -c path/to/gunicorn_conf.py path/to/settings.py

Where gunicorn_conf.py has this function:

def post_fork(server, worker):
    server.log.info("%s: Worker spawned" % worker.pid)

    from gunicorn.workers.sync import SyncWorker
    class SyncWorkerPreload(SyncWorker):
        def run(self):
            pass

        def init_process(self):
            super(SyncWorkerPreload, self).init_process()
            from django.db.models.loading import get_apps
            get_apps()
            server.log.info('%s: App loaded' % self.pid)
            super(SyncWorkerPreload, self).run()

    worker.__class__ = SyncWorkerPreload

Hope that helps someone google'ing to find this hint.

Pretty awesome that Gunicorn was flexible enough to allow for this!

UPDATE: updated code, simply loading modules in settings sometimes breaks due to the way django loads modules I assume... this new code monkey patches gunicorn to load the modules at a "safe" time... hopefully

UPDATE: gunicorn 0.12.1 fixes this issue

This does sound like it's due to Django's delayed imports on some modules. I'd either go with settings.py or maybe the specific app's urls.py to get them imported when the worker boots.