TypeError: expected string when using re.findall on web page text -- why?
I'm trying to learn how to screen-scrape with BeautifulSoup.
from urllib import urlopen
from BeautifulSoup import BeautifulSoup
import re
webpage = urlopen('http://feeds.feedburner.com/zenhabits').read()
patFinderTitle = re.compile('<h4 class="itemtitle"><a href=(.*)</a></h4>')
findPatTitle = re.findall(patFinderTitle,webpage)
listIterator = []
listIterator[:] = range(1, 5)
for i in listIterator:
print findPatTitle[i]
print("\n")
Error
Traceback (most recent call last):
File "//da-srv1/users/xxxxx/Desktop/fetcher", line 14, in <module>
print findPatTit开发者_开发技巧le[i]
**IndexError: list index out of range**
Use the following expression:
patFinderTitle.findall(webpage)
You can't do the equivalent of re.findall(re.compile(<expression>), <string>)
since re.findall
only accepts a regular expression as a string -- and re.compile(<expression>)
returns a compiled regex object. So you need to use your compiled regex object patFinderTitle
and call its findall()
method (see above).
EDIT: Oh. Turns out you can do re.findall(re.compile(<expression>), <string>)
. The more you know.
You left out the parenthesis on the read() function call, so webpage was a function and not a string.
webpage = urlopen('http://feeds.feedburner.com/zenhabits').read()
精彩评论