开发者

unicode().decode('utf-8', 'ignore') raising UnicodeEncodeError

Here is the code:

>>> z = u'\u2022'.decode('utf-8', 'ignore')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeEncodeError: 'latin-1' codec 开发者_如何学Gocan't encode character u'\u2022' in position 0: ordinal not in range(256)

Why is UnicodeEncodeError raised when I am using .decode?

Why is any error raised when I am using 'ignore'?


When I first started messing around with python strings and unicode, It took me awhile to understand the jargon of decode and encode too, so here's my post from here that may help:


Think of decoding as what you do to go from a regular bytestring to unicode and encoding as what you do to get back from unicode. In other words:

You de-code a str to produce a unicode string (in Python 2)

and en-code a unicode string to produce a str (in Python 2)

So:

unicode_char = u'\xb0'

encodedchar = unicode_char.encode('utf-8')

encodedchar will contain your unicode character, displayed in the selected encoding (in this case, utf-8).

The same principle applies to Python 3. You de-code a bytes object to produce a str object. And you en-code a str object to produce a bytes object.


From http://wiki.python.org/moin/UnicodeEncodeError

Paradoxically, a UnicodeEncodeError may happen when decoding. The cause of it seems to be the coding-specific decode() functions that normally expect a parameter of type str. It appears that on seeing a unicode parameter, the decode() functions "down-convert" it into str, then decode the result assuming it to be of their own coding. It also appears that the "down-conversion" is performed using the ASCII encoder. Hence an encoding failure inside a decoder.


You're trying to decode a unicode. The implicit encoding to make the decode work is what's failing.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜