Caching queries in Django

In a django project I only need to cache a few queries, using, because of server limitations, a cache table instead of memcached.

One of thos开发者_StackOverflowe queries looks like this:

Let's say I have a Parent object, which has a lot of Child objects. I need to store the result of the simple query parent.childs.all().

I have no problem with that, and everything works as expected with some code like

key = "%s_children" %(parent.name)
value = cache.get(key)
if value is None:
  cache.set(key, parent.children.all(), CACHE_TIMEOUT)
  value = cache.get(key)

But sometimes, just sometimes, the cache.set does nothing, and, after executing cache.set, cache.get(key) keeps returning None.

After some test, I've noticed that cache.set is not working when parent.children.all().count() has higher values.

That means that if I'm storing inside of key (for example) 600 children objects, it works fine,

but it wont work with 1200 children.

So my question is: is there a limit to the data that a key could store? How can I override it?

Second question: which way is "better", the above code, or the following one?

key = "%s_children" %(parent.name)
value = cache.get(key)
if value is None:
  value = parent.children.all()
  cache.set(key, value, CACHE_TIMEOUT)

The second version won't cause errors if cache.set doesn't work, so it could be a workaround to my issue, but obviously not a solution.

In general, let's forget about my issue, which version would you consider "better"?

Is your backend MySQL ?

In MySQL, TEXT field are limited to 65,000 bytes but django will use this type for the value field.

INSERT & UPDATE query will fail silently if your data si too big.

My advice is keep your data small. Storing a full QuerySet seems overkill. Can't you only store the required fields ?

Like this query

resultset = parent.children.values_list(*fields).all()

will fetch the needed fields, not the full instances.

I see three possibilities. The first seems most likely to me, since you've said that you're running with constrained resources:

1. It may be that it takes an extremely long time to fetch your query, encode it, then store it again - so long that your web server process runs out of memory or times out and dies before the data is saved.

2. Or, it may be that your database has some rather small limit on how long a text column can be, and storing 1200 base64 encoded pickled objects exceeds that limit.

3. Perhaps your cache key exceeds 255 characters.

Funky thing about memcached - it has a 1mb limit as to what can be stored in a given cache key. You can configure this limit, but it'll affect the portability of your code if you deploy on a server where this default is outside your control.

See "Storing Lists of Data" here: http://code.google.com/p/memcached/wiki/FAQ