Django + AppEngine counting more the 1000 records

I'm trying to count the number of entities in my db that were created between certain dates.

I know that google removed the limitation on fetching more then 1000 results per query but every time I try to use the remote api I get only 1000 re开发者_开发知识库sult count.

Does any one knows how can I count more the 1000 entities in order to check how much entities I have?

Thanks.

Netanel

The limitation has been removed in the more recent versions of the SDK, so you might want to check that out and upgrade.

This still won't completely solve your problem, though... issue is that with GAE count queries are a very bad idea - because this isn't a table (its more of a distributed array) whenever you do a count it actually has to hit (and maybe fetch, I'm not sure) each every one of the objects that match your query. This is a really expensive operation, so if you do it a lot you're using up resources unnecessarily. If you have enough objects you might even hit the 30 second request limit.

Take a look at other more scalable ways of doing this, like a map reduce (count and store the count for minute, then each hour, then day, then week, then month, etc) or a sharded counter.

Model.gql("WHERE creation_date >= :start_date AND creation_date <= :end_date", 
            start_date=startdate, end_date=enddate).count()

should return how many entities your query matches, even if you have more than 1000.

You are using at least v1.3.6 of the SDK, as this is where the count limitation was removed?