Decoding error in django filebrowser 3.2 on linux

The error message:

Traceback:
File "/web/hvita_perlan/lib/python2.6/site-packages/django/core/handlers/base.py" in get_response
  111.                         response = callback(request, *callback_args, **callback_kwargs)
File "/web/hvita_perlan/lib/python2.6/site-packages/django/contrib/admin/views/decorators.py" in _checklogin
  19.             return view_func(request, *args, **kwargs)
File "/web/hvita_perlan/lib/python2.6/site-packages/django/views/decorators/cache.py" in _wrapped_view_func
  79.         response = view_func(request, *args, **kwargs)
File "/web/hvita_perlan/lib/python2.6/site-packages/filebrowser/views.py" in browse
  99.         fileobject = FileObject(os.path.join(file_dir, file))
File "/web/hvita_perlan/lib/python2.6/posixpath.py" in join
  70.             path += '/' + b

Exception Type: UnicodeDecodeError at /admin/filebrowser/browse/
Exception Value: 'ascii' codec can't decode byte 0xc3 in position 3: ordinal not in range(128)

The string that could not be encoded/decode开发者_运维问答d was: /1h����.j

The file real filename is: 1hæð.jpg

some info:

> locale
LANG=en_GB.UTF-8

-

> python manage.py shell
>>> import locale
>>> locale.getlocale()
('en_GB', 'UTF8')

>>> import os
>>> os.stat('../uploads/_promotional/1hæð_fb_thumb.jpg')
posix.stat_result(st_mode=33279, st_ino=788504L, st_dev=51713L, st_nlink=1, st_uid=0, st_gid=0, st_size=1629L, st_atime=1311176542, st_mtime=1311176542, st_ctime=1311177235)

As you can see everything works in shell but not in django filebrowser.

It appears from looking at the docs that FileBrowser only supports ASCII.

It says in the exception:

Exception Value: 'ascii' codec can't decode byte 0xc3 in position 3: ordinal not in range(128)

os.path.join(file_dir, file) is getting a Unicode string, and it's being implicitly encoded to ASCII, rather than UTF-8. The unicode / string changes were made in Python 3 to remove this problem.

Somewhere, file_dir needs to be encoded with file_dir.encode('utf-8'). As a bad hack to make it work, you could try doing it in /web/hvita_perlan/lib/python2.6/site-packages/filebrowser/views.py on line 99:

fileobject = FileObject(os.path.join(file_dir.encode('utf-8'), file))

And then test, and repeat every time you find a new spot in FileBrowser that triggers this error.

In django-filebrowser 3.5.6 there is a setting FILEBROWSER_NORMALIZE_FILENAME if set to true in your settings.py it will make fb strip non standard characters from the file name. I had trouble finding info about it so posting it here enven though not sure if it works for the older version.

The solution is here: http://diveintopython.net/xml_processing/unicode.html

I solved the problem by adding sitecustomize.py to lib/python2.6/

# sitecustomize.py                   
# this file can be anywhere in your Python path,
# but it usually goes in ${pythondir}/lib/site-packages/
import sys
sys.setdefaultencoding('utf-8')

File browser files don't have the utf-8 header. I think they should change this. It looks like this:

#!/usr/bin/env python
# -*- coding: UTF-8 -*-

Its an old post but the issue remains.
I'm using django, apache2, django-filebrowser and get this Exception Value: 'ascii' codec can't decode byte 0xc3 in position...

What worked for me even using mod_wsgi.

#https://docs.djangoproject.com/en/1.2/howto/deployment/modpython/#if-you-get-a-unicodeencodeerror
#Put this in your apache2/envvars file.

export LANG='en_US.UTF-8'
export LC_ALL='en_US.UTF-8'

Hope it helps someone.