Non-english character getting changed to "." character in linux

I am trying to create a "Language File" for adding开发者_StackOverflow社区 localization to my Django application.

The problem is that when the translate text has some non-english characters then they get replaced by a "." character and Django complains this giving an error.

For eg:

English                        German 
Change Password                Passwort ändern

So here ä gets replaced by .

I don't understand why am I not able to paste this text inside my text file. I am able to paste it on the terminal but not inside the file when I open it through vi

I am using linux 2.6.32 on an embedded device.

The error message obtained in Django while preparing the language file is:

locale/de/LC_MESSAGES/django.po:472:70: invalid multibyte sequence

I have no idea why its happening like this. Can anybody suggest some solution?

Thanks in advance.

Okay, I solved my problem. I was getting the error invalid multibyte sequence because the problem was that the .po wasn't encoded properly.

I could confirm this by executing the following command:

file -i django.po

It gave the output that the encoding type of the file was ISO-8859-1

So, I converted the file to "UTF-8" using the following command:

iconv --from-code=ISO-8859-1 --to-code=UTF-8 django.po > django_utf8.po

Then replaced the old django.po file with the newly generated one and "makemessages" and "compilemessages" worked like a charm.

Thank you everybody above for your answer.

Hope this helps somebody in the future!

It seems you have some mismatch in encodings. Nowadays, Linux generally uses utf-8, but you may still have legacy iso-8859-n set somewhere and this would cause problems. Also django needs to know as which encoding to interpret the file.

• Vim has options encoding (that specifies encoding it uses internally and default for the other two), fileencoding (that specifies encoding to read and write particular file in (per-buffer)) and termencoding (that specifies encoding used to display on terminal; it's not used in the graphical version). It allows to specify fileencoding for file to open by adding ++enc= flag to the :edit command. So you open the file e.g. like :e ++enc=utf-8 file.txt. Try various combinations until the character shows up correctly and vim does not complain that something can't be converted. Than you'll know what encoding you are using.
• In python by default only ascii characters are translated between byte strings (str) and unicode (unicode) strings. If you want to open a file in different encoding and iterpret the strings, you have to open it with explicit encoding the codecs package.

If it's open somewhere in django, you'll have to find out how to tell it the encoding.