开发者

Beautiful Soup Unicode encode error

I am trying the following code with a particular HTML file

from BeautifulSoup import BeautifulSoup
import re
import codecs
impor开发者_运维技巧t sys
f = open('test1.html')
html = f.read()
soup = BeautifulSoup(html)
body = soup.body.contents
para = soup.findAll('p')
print str(para).encode('utf-8')

I get the following error:

UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 9: ordinal not in range(128)

How do I debug this?

I do not get any error when I remove the call to print function.


The str(para) builtin is trying to use the default (ascii) encoding for the unicode in para. This is done before the encode() call:

>>> s=u'123\u2019'
>>> str(s)
Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 3: ordinal not in range(128)
>>> s.encode("utf-8")
'123\xe2\x80\x99'
>>> 

Try encoding para directly, maybe by applying encode("utf-8") to each list element.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜