django is very slow

Some profiling shows template rendering as the culprit. (I'm trying on a page with ONLY cached queries.) But still, the template is very simple. the most complex part is a nested loop that runs for 10 times, but if everything goes well the nested loop doesn't run because it's cached. (like in my testing)

that is

{% for p in posts %}
 --{{p.by.username}}
 --{{p.text}}
 {% cache 600 p p.timestamp %}
    {% for img in p.images.all %}
      --{{img.path}}
    {% endfor %}
 {% endcache %}
{% endfor %}

I get ~80 req/s o开发者_如何学JAVAn the dev. server for this page. (I found I can multiply that number by 3 in the production deploy) For a comparison, I get 1000req/s for a trivial template that only contains a short static string.

Is that a known issue? How do I go about correcting/avoiding it?

In development mode, django does a lot of things to ease developing(for example: code reloading; template rendering for every request if template used; ...).

In production mode, deploy a WSGI server before django is preferred. Such wsgi might be gunicorn, uWSGI, ...

A typical production web server's layout might be: nginx -> gunicorn -> django

A simple comparison(django official tutorial code with a very simple "Hello World! @ time" template):

{% block content %}
    <p> Hello World! @ {{ now }}</p>
{% endblock %}

Development mode

run with django directly

python manage.py runserver

run apache-ab to test

ab -n1000 -c10 http://127.0.0.1:8000/polls/helloworld

Server Software:        WSGIServer/0.2 # django
Server Hostname:        127.0.0.1
Server Port:            8000

Document Path:          /polls/helloworld
Document Length:        59 bytes

Concurrency Level:      10
Time taken for tests:   3.438 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      242000 bytes
HTML transferred:       59000 bytes
Requests per second:    290.87 [#/sec] (mean)
Time per request:       34.380 [ms] (mean)
Time per request:       3.438 [ms] (mean, across all concurrent requests)
Transfer rate:          68.74 [Kbytes/sec] received

Production mode

run with gunicorn

../bin/gunicorn -w4 mysite.wsgi # with 4 workers

run apache-ab to test

ab -n1000 -c10 http://127.0.0.1:8000/polls/helloworld

Server Software:        gunicorn/19.7.1  # gunicorn
Server Hostname:        127.0.0.1
Server Port:            8000

Document Path:          /polls/helloworld
Document Length:        59 bytes

Concurrency Level:      10
Time taken for tests:   0.618 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      248000 bytes
HTML transferred:       59000 bytes
Requests per second:    1619.10 [#/sec] (mean)
Time per request:       6.176 [ms] (mean)
Time per request:       0.618 [ms] (mean, across all concurrent requests)
Transfer rate:          392.13 [Kbytes/sec] received

(Apparently I'm not "karmic" enough to post comments yet, or I would post this as a comment rather than an answer)

Could you elaborate on what you mean by "ONLY cached queries"?

Aside from that, it seems to me that your problem might be that you're hitting your database a lot during template rendering.

{% for p in posts %}
 --{{p.by.username}} {# 1 #}
 --{{p.text}}
 {% cache 600 p p.timestamp %}
    {% for img in p.images.all %} {# 2 #}
      --{{img.path}}
    {% endfor %}
 {% endcache %}
{% endfor %}

You provide "posts" to your template; that's one query, which you've said has 100 results.

For each iteration over posts, then, you are hitting the database at {# 1 #} to get p.by, which I assume to be a ForeignKey to an auth.User.

In addition to that, with an invalid cache (first run), you are hitting the db again at {# 2 #} to get the list of that post's images.

So for 100 items, you're hitting the database 201 times per request for an initial run, and 101 with a filled cache for the images loop.

Try using select_related with your posts query to pull these extra results in on the one query, if possible. Something like posts = Post.objects.select_related('by', 'images').filter(...) might do the trick, though I know select_related has limits when it comes to reverse foreign keys and m2m fields (it might not work for the images, depending on your models structure).