Force manage.py to output UTF-8 to avoid UnicodeEncodeErrors

[anonymous]

Got the following Unicode error while trying to run: ./manage sqlall ws with the workstyle-py application.
(Btw the application is writen for Django-0.91 and i found this error while porting it to Django-trunk)

The application tries to insert some Japanese (UTF-8) data into the database while running sqlall.
But the console running the ./manage.py command default to ASCII not UTF-8, thus raises an UnicodeEncodeError.
I figure that since Django should be full unicode now this should work and print out UTF-8 (or at least not raise an EncodeError)

The attached path forces the output print stream into UTF-8.
Hopefully this helps other people with Unicode problems on the console with SQL.

Traceback (most recent call last):
  File "./manage.py", line 11, in ?
    execute_manager(settings)
  File "/var/www/site/webapp/__init__.py", line 274, in execute_manager
    
  File "/var/www/site/webapp/__init__.py", line 224, in execute
    
  File "/usr/lib/python2.4/site-packages/django/core/management/base.py", line 71, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/lib/python2.4/site-packages/django/core/management/base.py", line 94, in execute
    print(txt)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 66-68: ordinal not in range(128)

[anonymous]

FYI: On the console of my workstation this error does not get raised and utf-8 get's printed correctly.
But when logging in on a remote server (with ssh) i need this (or a similar) patch to make the project work with manage sql* commands

[Russell Keith-Magee]

This is a very old report, with a diff that no longer applies, without a test case that actually describes how the problem can actually be replicated (I have no idea what workstyle-py is, and the report doesn't tell me where I can get it from).

I'm also reasonably certain that this has been resolved as a side effect of other changes (i.e., I've fixed other bugs about unicode in management output, like #12849). Closing worksforme; please reopen if you have a concrete example that fails.

[Joni]

This doesn't work for me with django 1.3.1, I get this error:

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 0: ordinal not in range(128)

#coding=utf-8
from django.core.management.base import BaseCommand

class Command(BaseCommand):
    def handle(self, *args, **options):
        self.stdout.write(u'á')

[Aymeric Augustin]

Django doesn't have enforce an output encoding, Python provides everything you need to get output in your teminal's charset!

You must set either your locale, or the ​PYTHONIOENCODING environment variable, according to your terminal's encoding — in most cases, utf-8.

Here's a demo:

% cat test.py
# coding: utf-8
print u"café!"


% unset LANG
% unset PYTHONIOENCODING
% python test.py
Traceback (most recent call last):
  File "test.py", line 3, in <module>
    print u"café!"
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 3: ordinal not in range(128)


% unset LANG
% export PYTHONIOENCODING=utf-8
% python test.py
café!


% export LANG=en_US.UTF-8
% unset PYTHONIOENCODING
% python test.py
café!

If you still get errors or mangled output, it means that Django is outputting encoded data instead of unicode. If this happens, it's a bug, please open a separate ticket.