Opened 10 years ago

Closed 10 years ago

Last modified 10 years ago

#1945 closed defect (worksforme)

Problem following the tutorial when using non English characters

Reported by: gradha@… Owned by: Jacob
Component: Internationalization Version:
Severity: minor Keywords:
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:


I'm following and its second part. Everything goes OK. Just, to try things, I modify most of the examples of code. So when I modified the polls/ to create the Poll object, instead of using pub_date's tutorial provided description, I wrote:

pub_date = models.DateTimeField('Fecha de publicación')

To make the script a valid python script, I added at the top:

# -*- coding: latin1 -*-

So later, much much later, when I'm adding the administrative interface, I reach the point where I have added the "class Admin: pass" to Poll and web browsed my way to see the first poll. Here, the poll is displayed, and the date description shows a broken character (the accented o): "Fecha de publicaci�n:" (not sure if that will get through, it shows a question mark inside a rotated square).

Well, I guess maybe Django wants unicode characters or stuff. So I modified my Poll class to read (note the unicode string):

pub_date = models.DateTimeField(u'Fecha de publicación')

Reloading the admin page I get:

UnicodeDecodeError at /admin/polls/poll/1/
'ascii' codec can't decode byte 0xc3 in position 75: ordinal not in range(128)

...along with a hell long of a callstack and environment dump. Since I'm used to these kind of things, I tried the following change:

pub_date = models.DateTimeField(u'Fecha de publicación'.encode("utf8"))

...which worked and didn't crash, but strikes me as unusually cumbersome to do. AFAICS the model doesn't properly treat unicode strings and the reason I get this OK with the utf8 encoding is just because that's what the output page charset is, and the result is processed by the browser. I guess if I switched that to latin1 now it would break visually as well, right? Is Django unable to handle internationalised stuff in the model?

If this is the case, I would add a note on the kind of strings you can use for the model.

Change History (5)

comment:1 Changed 10 years ago by asmodai@…

I cannot reproduce this.

I used:

# -*- coding: latin1 -*-

with a verbose name of:

models.CharField('áóéíàèìò', maxlength = 64)

and in my admin I see:


Did you check your browser to see if it selected/auto-detected the right page encoding? It should be UTF-8.

Note that I am using the latest SVN version and you didn't specify yours though.

comment:2 Changed 10 years ago by Adrian Holovaty

Resolution: fixed
Status: newclosed

Looks like this isn't a bug. (See previous comment.)

comment:3 Changed 10 years ago by Ramiro Morales

Component: DocumentationInternationalization
Resolution: fixed
Status: closedreopened

I'm reopening this ticket because I'm experiencing the same issue and to post some additional information:

SVN revision 3065

first line of has:

  # -*- coding: iso-8859-1 -*-

a model:

  class Project(models.Model):
    description = models.CharField('descripcion', blank=True, maxlength=255, db_column='DESCRIPTION')
    class Admin:
        fields = ((None, { 'fields': ('title', 'description', 'client', 'start_date', 'leader', 'status')} ), )
        list_display=('title', 'client', 'status', 'leader')

Django conf/ has


Project has

  DEBUG = True
  LANGUAGE_CODE = 'es-ar'
$ file ASCII English text

If I edit the field verbose name to add a non-ASCII char

    description = models.CharField('descripción', blank=True, maxlength=255, db_column='DESCRIPTION')
$ file ISO-8859 English text

And then whe trying to access the edit page for that model by selecting it from the change list page of the admin app, I get a 'Descripci�n:' field label.

I'm using the bundled Django development Web server.

Server-side info:
Linux, python 2.3.5

Client-side info:

Mozilla Firefox / Win32 / Windows 2000 + SP4 English

View -> Character Encoding -> UTF-8 is checked, if I change it to Western (ISO-8859-1) the
problem goes away.

When I change the verbose name to include the u prefix

    description = models.CharField(u'descripción', blank=True, maxlength=255, db_column='DESCRIPTION')

I also get the exception + back trace debug page:

Request Method:  	GET
Request URL: 	http://<my server DNS name>:8000/admin/intro/project/81/
Exception Type: 	UnicodeDecodeError
Exception Value: 	'ascii' codec can't decode byte 0xc3 in position 222: ordinal not in range(128)
Exception Location: 	/usr/lib/python2.3/site-packages/django/template/ in render, line 686
Traceback (most recent call last):
File "/usr/lib/python2.3/site-packages/django/template/" in render_node
  701. result = node.render(context)
File "/usr/lib/python2.3/site-packages/django/template/" in render
  113. nodelist.append(node.render(context))
File "/usr/lib/python2.3/site-packages/django/template/" in render
  115. return nodelist.render(context)
File "/usr/lib/python2.3/site-packages/django/template/" in render
  686. return ''.join(bits)

  UnicodeDecodeError at /admin/intro/project/81/
  'ascii' codec can't decode byte 0xc3 in position 222: ordinal not in range(128)

Why is the text getting decoded with the ascii codec?.

Feel free to ask for further info bacause I have the environment at hand.




comment:4 Changed 10 years ago by Ramiro Morales

More info:

"Request information - META" section of the backtrace:

HTTP_COOKIE'sessionid=145006e2d5e41bb0ae6e740c5c3e4811; SITESERVER=ID=f9aa77d7c40371371daebd95787b0042'
HTTP_HOST'<my Django server DNS name>:8000'
HTTP_IF_MODIFIED_SINCE'Fri, 02 Jun 2006 18:46:50 GMT'
HTTP_REFERER'http://<my Django server DNS name>:8000/admin/intro/project/'
HTTP_USER_AGENT'Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv: Gecko/20060508 Firefox/'
REMOTE_ADDR'<my workstation IP address>'
REMOTE_HOST'<my workstation DNS name>'
SERVER_NAME'<my Django server DNS name>'
SERVER_SOFTWARE'WSGIServer/0.1 Python/2.3.5'
wsgi.errors<open file '<stderr>', mode 'w' at 0xb7d890a0>
wsgi.file_wrapper<class django.core.servers.basehttp.FileWrapper at 0xb7a9e7dc>
wsgi.input<socket._fileobject object at 0xb7031c34>
wsgi.version(1, 0)

comment:5 Changed 10 years ago by Ramiro Morales

Resolution: worksforme
Status: reopenedclosed

I'm seeing now that a solution to this problem was posted in one of the lasts comments of ticket #170 (I've tested it and it works).

So I'm closing this ticket again. Perhaps I will open another targeted at the documentation component because it seems a somewhat common pitfall for non English users (see #1715, If the recommended solution is in fact the right one then maybe a note could be added to the models docs stating that for non ASCII languages it is beter to encode the file in UTF-8 and flag it so by using magic comments as per PEP 0263


Note: See TracTickets for help on using tickets.
Back to Top