App Engine: CPU Over Quota on urlfetch()

Hey. I'm quite new to App Engine. I created a web-based Twitter app which is now running on App Engine and I'm constantly hitting my CPU Over Quota limits. I did a little profiling and I found out that every request consists of two urlfetch queries, each one of which takes up to 2 CPU seconds. That time is probably spent waiting, all the rest of the code is done in under 200 ms (including work with the Datastore). The Quota is for 6.5 hours per day and every request of mine takes approx. 4 CPU seconds. I ran out of the free quota this morning in only a few hours.

What is the way around this? I can't make Twitter respond to my API calls quicker, and I cannot cache the results, since every request is fo开发者_如何学Pythonr a different Twitter profile.

Any help is appreciated, Thanks!

I would find it confusing that time spent in urlfetch to wait for a remote response is counted towards your CPU quota, given that there is no CPU time spent.

But assuming that's really the problem, asynchronous requests may be your solution. At a minimum, you can overlap the two urlfetch requests to proceed simultaneously. Perhaps you find other stuff you can do until the response is back.

You should change the design of your application.

Instead of making requests to Twitter from App Engine for every user request:

• Do the request in the user's browser with JavaScript if possible.
• After a urlfetch, store Twitter's response in the datastore, since a call to the datastore is faster on the next request. If you can cache something in memcache, even better.
• Update the stored data regularly with the help of cron jobs and task queue.