Testing for UnicodeDecodeError in Python 3
I have the following test for a function that can only accept unicode text in Python 2.x
def testNonUnicodeInput(self):
""" Test falure on non-unicode input. """
input = "foo".encode('utf-16')
self.assertRaises(UnicodeDecodeError, myfunction, input)
However, that test fails when run in Python 3.x. I get:
AssertionError: UnicodeDecodeError not raised by myfunction
I'm trying to figure out how to set up a test that will continue to work in Python 2.x, but will also work after being run through 2to3 on Python 3.x.
I should probably note that I'm doing the following in my function to force unicode:
def myfunction(input):
""" myfunction only accepts unicode input. """
...
try:
source = unicode(source)
except UnicodeDecodeError, e:
# Customise error message while maintaining original trackback
e.reason += '. -- Note: Myfunction only accepts unicode input!'
raise
...
Of course, that (along with the test) is being run through 2to3 before being run on Python 3.x. I suppose what I actually want on Python 3 is to not accept byte strings which I tho开发者_如何学Cugh I was doing by encoding the string first. I didn't use 'utf-8' as the encoding because I understand that that is the default.
Anyone have any ideas for consistency here?
You shouldn't have to do anything to Python 3 strings; they're all Unicode. Just test isinstance(s, str). Or, if the problem is the other way around, you'd want to use bytes.decode().
Okay, a way to cause UnicodeDecodeError
in both Python 3 and Python 2:
Python 3:
>>> "foo".encode('utf-16').decode('utf-8')
Traceback (most recent call last):
File "<pyshell#61>", line 1, in <module>
"foo".encode('utf-16').decode('utf-8')
UnicodeDecodeError: 'utf8' codec can't decode byte 0xff in position 0: unexpected code byte
Python 2:
>>> "foo".encode('utf-16').decode('utf-8')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python26\lib\encodings\utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xff in position 0: unexpected code byte
Not sure if 2to3
would automatically convert the string literal to the b"foo"
syntax, though. If it does, you'd just have to take out the b
manually, or set it to ignore that somehow.
Well, I've decided to just skip the test under Python 3 for now.
if sys.version_info < (3, 0):
input = "foo".encode('utf-16')
self.assertRaises(UnicodeDecodeError, myfunction, input
However, if someone could suggest a test that would pass under Python 2 & 3, I'm open to suggestions.
精彩评论