DEADLOCK_WRAP error when using Berkeley Db in python (bsddb)
I am using a berkdb to store a huge list of key-value pairs but for some reason when i try to access some of the data later i get this error:
try:
key = 'scrape011201-590652'
contenttext = contentdict[key]
except:
print the error
<type 'exceptions.KeyError'> 'scrape011201-590652' in
contenttext = contentdict[key]\n', ' File "/usr/lib64/python2.5/bsddb/__init__.py",
line 223, in __getitem__\n return _DeadlockWrap(lambda: self.db[key]) #
开发者_StackOverflowself.db[key]\n', 'File "/usr/lib64/python2.5/bsddb/dbutils.py", line 62, in
DeadlockWrap\n return function(*_args, **_kwargs)\n', ' File
"/usr/lib64/python2.5/bsddb/__init__.py", line 223, in <lambda>\n return
_DeadlockWrap(lambda: self.db[key]) # self.db[key]\n']
I am not sure what DeadlockWrap is but there isnt any other program or process accessing the berkdb or writing to it (as far as i know,) so not sure how we could get a deadlock, if its referring to that. Is it possible that I am trying to access the data to rapidly? I have this function call in a loop, so something like
for i in hugelist:
#try to get a value from the berkdb
#do something with it
I am running this with multiple datasets and this error only occurs with one of them, the largest one, not the others.
I'm pretty certain the DeadlockWrap
stuff is not relevant here. It's simply a way to automagically provide retries with a back-off strategy. In other words, if the database manipulation fails, it waits a little bit then tries again, a number of times before finally failing.
You seem to be getting a KeyError
from your dictionary get
operation which is more likely to be due to the fact that the key you're using doesn't actually exist in the database.
Try your code with something like:
try:
key = 'scrape011201-590652'
if not contentdict.has_key(key):
print "Urk!, No record for %s"%(key)
contenttext = contentdict[key]
except:
print the error
This should show you if the record doesn't exist in the table (by outputting the Urk!
message). As to what you do in that case, it depends on your architecture. You would probably want to return either None
or an empty string. You may also want to do exactly what you're doing now (raising an exception).
contenttext = contentdict[key] if contentdict.has_key(key) else None
精彩评论