Python, Django and pyodbc: invalid characters
I am connecting to a MS SQL server database using pyodbc. The error im getting is the following
invalid byte sequence for encoding "UTF8": 0x93
HINT: This error can also happen if the byte sequence does not match the
encoding expected by the server, which is controlled by "client_encoding".
The SQL database is encoded using Latin1 and I am using postgres with django, which expects UTF8.
I am very new to using pyodbc and cannot solve this problem. i have attempted to filter through piles of google searches but with no luck. Some help would be greatly appreciated.
EDIT
The Postgres db is the main db for the project. I want to be able to pull data from the SQL Server. This process will not be done often though...
The point at which the error occurs is from the read开发者_JAVA百科 from the SQL Server db
You have given next to no clues but a reasonable guess is:
You need to decode your MS SQL Server data to unicode using the correct encoding, and (not necessarily immediately) encode it as 'UTF-8' for transmission to postgres.
What makes you think that the encoding used on the SQL Server db is latin1
and not cp125x
? True latin1
on an MS product is highly unlikely. Your errant byte '\x93'
when decoded as cp1252
(the usual suspect) gives U+201C LEFT DOUBLE QUOTATION MARK, commonly used in e.g. MS Word and commonly found pasted into data which ends up in a db. Decoding as latin1
produces U+0093 which is some arcane control character whose usage in practice is as rare as hens' teeth.
精彩评论