Why is this code so CPU and memory intensive?
Why does this code takes 0.32 CPU hours and has an avreage memory of 24.6 MBytes? The page refreshes about 30 times until it stops because there are no more datastore entities.
class MainHandler(webapp.RequestHandler):
def get(self):
found = False
q =开发者_开发问答 MyModel.all(keys_only=True).fetch(1000)
if len(q):
self.response.out.write("Deleted %d MyModel entries" % len(q))
found = True
db.delete(q)
q = MyModel2.all(keys_only=True).fetch(1000)
if len(q):
self.response.out.write("Deleted %d MoModel2 entries" % len(q))
found = True
db.delete(q)
if found:
self.response.out.write('<meta http-equiv="Refresh" content="0"/>')
def main():
application = webapp.WSGIApplication([('/', MainHandler)], debug=True)
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
Is there something I can do to speed it up and use less memory?
Thank you
Seconding task queues. Using mapreduce to delete entities is a common practice. It'll manage the tasks for you.
A micro optimization is to make one webapp.WSGIApplication
and reuse it from a class variable, rather than creating a new one every time main()
is invoked.
I found out by testing that I can lower the CPU usage by 3 times only by fetching 200 entries at a time:
q = MyModel.all(keys_only=True)
for i in xrange(0, 1000, 200):
db.delete(q.fetch(200))
Fetching 100 results at a time actually is slower than fetching 200 at a time. But I'll check into the mapreduce, but I don't think that this is a thing
精彩评论