TextField takes unicode in Django

I am trying to save a string contains characters whose ordinal is not in range(128) into database. The field is declared as TextField in model and when I called the save() method, an excep开发者_Go百科tion was thrown:

File "/Library/Python/2.6/site-packages/django/db/models/base.py", line 434, in save self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/Library/Python/2.6/site-packages/django/db/models/base.py", line 500, in save_base rows = manager.using(using).filter(pk=pk_val)._update(values)
File "/Library/Python/2.6/site-packages/django/db/models/query.py", line 491, in _update return query.get_compiler(self.db).execute_sql(None)
File "/Library/Python/2.6/site-packages/django/db/models/sql/compiler.py", line 861, in execute_sql cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/Library/Python/2.6/site-packages/django/db/models/sql/compiler.py", line 717, in execute_sql sql, params = self.as_sql()
File "/Library/Python/2.6/site-packages/django/db/models/sql/compiler.py", line 826, in as_sql val = field.get_db_prep_save(val, connection=self.connection)
File "/Library/Python/2.6/site-packages/django/db/models/fields/subclassing.py", line 28, in inner return func(*args, **kwargs)
File "/Library/Python/2.6/site-packages/django/db/models/fields/subclassing.py", line 28, in inner return func(*args, **kwargs)
File "/Library/Python/2.6/site-packages/django/db/models/fields/__init__.py", line 276, in get_db_prep_save return self.get_db_prep_value(value, connection=connection, prepared=False)
File "/Library/Python/2.6/site-packages/django/db/models/fields/subclassing.py", line 53, in inner return func(*args, **kwargs)
File "/Library/Python/2.6/site-packages/django/db/models/fields/__init__.py", line 652, in get_db_prep_value value = self.get_prep_value(value)
File "/Library/Python/2.6/site-packages/django/db/models/fields/__init__.py", line 647, in get_prep_value return self.to_python(value)
File "/Library/Python/2.6/site-packages/django/db/models/fields/__init__.py", line 610, in to_python if not ansi_date_re.search(value):
TypeError: expected string or buffer

Here is what I did:

str_unicode = str.encode('utf-8')
m = MyModel.new(str = str_unicode)
m.save()

So what should I do to enable the unicode compatibility?

Updated: I am currently using sqlite3 for development, python 2.6.1 and Django 1.2.4 on Mac

utf-8 is not unicode! This line:

str_unicode = str.encode('utf-8')

does the opposite of what you intended. It takes unicode, and converts it to utf-8. Don't do this.

Your problem is not with the TextField at all. If you look at the code referenced in the traceback, you're triggering an exception from within DateField's to_python method. You can see the relevant code here. Basically it looks like you're passing something inappropriate as the input to another field on your model.

Lesson to be learned: ALWAYS READ THE TRACEBACK CAREFULLY!

As a side note, if you really want to coerce something to unicode in Python call the builtin unicode type:

str_unicode = unicode(str)

Using the encode method to convert to UTF-8 does something completely different.

Django works fine with unicode.

But if your input are not properly converted you will have trouble. As an exemple : I have had a lot of trouble with an old MyISAM database populated wrongly by a bugged php script. Check your input.

What database backend do you use? What level of python ?

The reason is I didn't load mysql settings in django's database configuration, which is not related to unicode problem I presented.

The configuration should be like this:

   DATABASES = {
       'default': {
           'ENGINE': 'django.db.backends.mysql', 
           'NAME': 'db',
           'USER': 'xxx',
           'PASSWORD': 'xxx', 
           'HOST': 'localhost',
           'PORT': '3306', 
           'OPTIONS' : {
              "init_command": "SET storage_engine=INNODB",
              'read_default_file': '/etc/my.cnf',
           }
       }
   }