Opened 11 years ago

Closed 7 years ago

#5877 closed Uncategorized (wontfix)

Force to output UTF-8 to avoid UnicodeEncodeErrors

Reported by: anonymous Owned by: nobody
Component: Core (Management commands) Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Design decision needed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


Got the following Unicode error while trying to run: ./manage sqlall ws with the workstyle-py application.
(Btw the application is writen for Django-0.91 and i found this error while porting it to Django-trunk)

The application tries to insert some Japanese (UTF-8) data into the database while running sqlall.
But the console running the ./ command default to ASCII not UTF-8, thus raises an UnicodeEncodeError.
I figure that since Django should be full unicode now this should work and print out UTF-8 (or at least not raise an EncodeError)

The attached path forces the output print stream into UTF-8.
Hopefully this helps other people with Unicode problems on the console with SQL.

Traceback (most recent call last):
  File "./", line 11, in ?
  File "/var/www/site/webapp/", line 274, in execute_manager
  File "/var/www/site/webapp/", line 224, in execute
  File "/usr/lib/python2.4/site-packages/django/core/management/", line 71, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/lib/python2.4/site-packages/django/core/management/", line 94, in execute
UnicodeEncodeError: 'ascii' codec can't encode characters in position 66-68: ordinal not in range(128)

Attachments (1)

patch-django-core-management-base_unicode_printing_#5877.patch (798 bytes) - added by anonymous 11 years ago.

Download all attachments as: .zip

Change History (6)

comment:1 Changed 11 years ago by anonymous

FYI: On the console of my workstation this error does not get raised and utf-8 get's printed correctly.
But when logging in on a remote server (with ssh) i need this (or a similar) patch to make the project work with manage sql* commands

comment:2 Changed 11 years ago by Simon G <dev@…>

Summary: UnicodeEncodeError in sqlall on project with utf8 dataForce to output UTF-8 to avoid UnicodeEncodeErrors
Triage Stage: UnreviewedDesign decision needed

comment:3 Changed 8 years ago by Russell Keith-Magee

Resolution: worksforme
Status: newclosed

This is a very old report, with a diff that no longer applies, without a test case that actually describes how the problem can actually be replicated (I have no idea what workstyle-py is, and the report doesn't tell me where I can get it from).

I'm also reasonably certain that this has been resolved as a side effect of other changes (i.e., I've fixed other bugs about unicode in management output, like #12849). Closing worksforme; please reopen if you have a concrete example that fails.

comment:4 Changed 7 years ago by Joni

Easy pickings: unset
Resolution: worksforme
Severity: Normal
Status: closedreopened
Type: Uncategorized
UI/UX: unset

This doesn't work for me with django 1.3.1, I get this error:

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 0: ordinal not in range(128)

from import BaseCommand

class Command(BaseCommand):
    def handle(self, *args, **options):

comment:5 Changed 7 years ago by Aymeric Augustin

Resolution: wontfix
Status: reopenedclosed

Django doesn't have enforce an output encoding, Python provides everything you need to get output in your teminal's charset!

You must set either your locale, or the PYTHONIOENCODING environment variable, according to your terminal's encoding — in most cases, utf-8.

Here's a demo:

% cat
# coding: utf-8
print u"café!"

% unset LANG
% python
Traceback (most recent call last):
  File "", line 3, in <module>
    print u"café!"
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 3: ordinal not in range(128)

% unset LANG
% python

% export LANG=en_US.UTF-8
% python

If you still get errors or mangled output, it means that Django is outputting encoded data instead of unicode. If this happens, it's a bug, please open a separate ticket.

Note: See TracTickets for help on using tickets.
Back to Top