开发者

Why does my function "hangs"

def retCursor():
    host = "localhost"
    user = "disappearedng"

    db = "gupan_crawling3"
    conn = MySQLdb.connect( host=host, user=user, passwd=passwd, db=db)
    cursor = conn.cursor()
    return cursor
singleCur = retCursor()


def checkTemplateBuilt(netlocH):
    """Used by crawler specifically, this check directly whether template has been built"""
    singleCur.execute( """SELECT templateBuilt FROM templateEnough WHERE netloc=%s""", [ netlocH])
    r = singleCur.fetchone()
    if r:
        if bool( r[0]):
            return True
    return False

Hi everyone I am currently using MySQLdb. For some reason, after perhaps 30 mins of running my app comes to a complete halt. It appears that this function is blocking me. (DOn't know for what reason)

Traceback (most recent call last):
  File "/usr/lib/python2.6/multiprocessing/process.py", line 231, in _bootstrap
    self.run()
  File "/mount/950gb/gupan5/disappearedng_temp/code_temp_NEWBRANCH/gupan5-yah开发者_运维百科oo/crawling/templateCrawling/TemplateCrawler/crawler/crawler.py", line 117, in run
    self.get_check_put()
  File "/mount/950gb/gupan5/disappearedng_temp/code_temp_NEWBRANCH/gupan5-yahoo/crawling/templateCrawling/TemplateCrawler/crawler/crawler.py", line 66, in get_check_put
    if not self.checkLinkCrawlability(linkS, priority):
  File "/mount/950gb/gupan5/disappearedng_temp/code_temp_NEWBRANCH/gupan5-yahoo/crawling/templateCrawling/TemplateCrawler/crawler/crawler.py", line 53, in checkLinkCrawlability
    if checkTemplateBuilt( getNetLoc( link)):
  File "/mount/950gb/gupan5/disappearedng_temp/code_temp_NEWBRANCH/gupan5-yahoo/crawling/templateCrawling/TemplateCrawler/publicapi/publicfunc.py", line 71, in checkTemplateBuilt
    singleCur.execute( """SELECT templateBuilt FROM templateEnough WHERE netloc=%s""", [ netlocH])
  File "/var/lib/python-support/python2.6/MySQLdb/cursors.py", line 153, in execute
    r = self._query(query)
KeyboardInterrupt

Btw this is the table:

CREATE TABLE templateEnough( 
    `netloc` INT(32) unsigned NOT NULL,   
    `count` SMALLINT(32) unsigned NOT NULL,
    `templateBuilt` TINYINT(1) unsigned DEFAULT 0 NOT NULL,
    PRIMARY KEY ( netloc )
) ENGINE=MEMORY DEFAULT CHARSET=utf8
;

Any ideas?


There might be a lock on the table preventing the query from completing.


Try logging the query string to a file right before you execute it. Then when you think it is hung, you can look at the query and see if it works manually


According to your traceback, you interrupted the script during the execution of checkTemplateBuilt, not enoughPassedForTemplate.

I think the problem lies in a different part of the code; maybe there is an infinite loop somewhere? Maybe in the run function?

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜