Please find below the piece of code that I use to tokenize a string. strList = list(token[STRING] for token in generate_tokens(StringIO(line).readline) if token[STRING])
What is the best way to write the contents of a StringIO buffer to a file ? I currently do something like:
I\'m getting a UnicodeEncodeError: \'ascii\' codec can\'t encode character u\'\\xa3\' in position 34: ordinal not in range(128)