Why does my function "hangs"
def retCursor():
host = "localhost"
user = "disappearedng"
db = "gupan_crawling3"
conn = MySQLdb.connect( host=host, user=user, passwd=passwd, db=db)
cursor = conn.cursor()
return cursor
singleCur = retCursor()
def checkTemplateBuilt(netlocH):
"""Used by crawler specifically, this check directly whether template has been built"""
singleCur.execute( """SELECT templateBuilt FROM templateEnough WHERE netloc=%s""", [ netlocH])
r = singleCur.fetchone()
if r:
if bool( r[0]):
return True
return False
Hi everyone I am currently using MySQLdb. For some reason, after perhaps 30 mins of running my app comes to a complete halt. It appears that this function is blocking me. (DOn't know for what reason)
Traceback (most recent call last):
File "/usr/lib/python2.6/multiprocessing/process.py", line 231, in _bootstrap
self.run()
File "/mount/950gb/gupan5/disappearedng_temp/code_temp_NEWBRANCH/gupan5-yah开发者_运维百科oo/crawling/templateCrawling/TemplateCrawler/crawler/crawler.py", line 117, in run
self.get_check_put()
File "/mount/950gb/gupan5/disappearedng_temp/code_temp_NEWBRANCH/gupan5-yahoo/crawling/templateCrawling/TemplateCrawler/crawler/crawler.py", line 66, in get_check_put
if not self.checkLinkCrawlability(linkS, priority):
File "/mount/950gb/gupan5/disappearedng_temp/code_temp_NEWBRANCH/gupan5-yahoo/crawling/templateCrawling/TemplateCrawler/crawler/crawler.py", line 53, in checkLinkCrawlability
if checkTemplateBuilt( getNetLoc( link)):
File "/mount/950gb/gupan5/disappearedng_temp/code_temp_NEWBRANCH/gupan5-yahoo/crawling/templateCrawling/TemplateCrawler/publicapi/publicfunc.py", line 71, in checkTemplateBuilt
singleCur.execute( """SELECT templateBuilt FROM templateEnough WHERE netloc=%s""", [ netlocH])
File "/var/lib/python-support/python2.6/MySQLdb/cursors.py", line 153, in execute
r = self._query(query)
KeyboardInterrupt
Btw this is the table:
CREATE TABLE templateEnough(
`netloc` INT(32) unsigned NOT NULL,
`count` SMALLINT(32) unsigned NOT NULL,
`templateBuilt` TINYINT(1) unsigned DEFAULT 0 NOT NULL,
PRIMARY KEY ( netloc )
) ENGINE=MEMORY DEFAULT CHARSET=utf8
;
Any ideas?
There might be a lock on the table preventing the query from completing.
Try logging the query string to a file right before you execute it. Then when you think it is hung, you can look at the query and see if it works manually
According to your traceback, you interrupted the script during the execution of checkTemplateBuilt
, not enoughPassedForTemplate
.
I think the problem lies in a different part of the code; maybe there is an infinite loop somewhere? Maybe in the run
function?
精彩评论