Google App Engine Task Deadline
is there a way to handle any soft deadline while executing a task ? The DeadlineExceededError is thrown after 10 minutes of execution and I am given few seconds after that to do some stuff. I want to clean up somethings before the task dies and create a new task. This may take few seconds. Is there a way to do this by capturing any exception like around 9 mins. I know I can manually throw an exception after 9 mins. But can this be done automatically by GAE ?
class FillMtxHandler():
def post(self,index,user,seqlen):
try :
FillMtx(index,user,seqlen)
except DeadlineExceededError:
deferred.defer(self.post,index,user,seqlen)
The above is my code. index is a list and starts from 0. It will be incremented inside FillMtx. Once a deadline exceeded error is thrown, i want to continue from where index was incremented last. I get the following error
The API call taskqueue.BulkAdd() was explicitly cancelled.
Traceback (most recent call last):
File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 517, in __call__
handler.post(*groups)
File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 258, in post
run(self.request.body)
File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 124, in run
return func(*args, **kwds)
File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 166, in invoke_member
return getattr(obj, membername)(*args, **kwargs)
File "/base/data/home/apps/0xxopdp/3.347813391084738922/fillmtx.py", line 204, in post
deferred.defer(self.post,index,user,seqlen)
File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 241, in defer
return task.add(queue, transactional=transactional)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/taskqueue/taskqueue.py", line 688, in add
return Queue(queue_name).add(self, transactional=transactional)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/taskqueue/taskqueue.py", line 744, in add
self.__AddTasks(tasks, transactional)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/taskqueue/taskqueue.py", line 770, in __AddTasks
apiproxy_stub_map.MakeSyncCall('taskqueue', 'BulkAdd', request, 开发者_开发知识库response)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 86, in MakeSyncCall
return stubmap.MakeSyncCall(service, call, request, response)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 286, in MakeSyncCall
rpc.CheckSuccess()
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_rpc.py", line 126, in CheckSuccess
raise self.exception
CancelledError: The API call taskqueue.BulkAdd() was explicitly cancelled.
I find that a new task has been created and queued up. But why does GAE throw this error still?
You can't control when you get the soft deadline exceeded error. Instead, you should use your own timer (take the wallclock time when you start, and compare it to the current time on each trip around your main loop), and abort when you're near enough the limit that you want to stop.
You don't have to raise an exception after 9 minutes; when the soft deadline exception raises, you have enough time to add a clean-up task to the Task Queue via deferred.
from google.appengine.ext import deferred
...
try:
# Do your stuff
except DeadlineExceededError:
deferred.defer(do_your_cleanup, ..)
In this way you have 10 minutes to do any clean-up stuff you need in your app.
精彩评论