#1945 closed defect (worksforme)
Problem following the tutorial when using non English characters
Reported by: | Owned by: | Jacob | |
---|---|---|---|
Component: | Internationalization | Version: | |
Severity: | minor | Keywords: | |
Cc: | Triage Stage: | Unreviewed | |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
I'm following http://www.djangoproject.com/documentation/tutorial1/ and its second part. Everything goes OK. Just, to try things, I modify most of the examples of code. So when I modified the polls/models.py to create the Poll object, instead of using pub_date's tutorial provided description, I wrote:
pub_date = models.DateTimeField('Fecha de publicación')
To make the script a valid python script, I added at the top:
# -*- coding: latin1 -*-
So later, much much later, when I'm adding the administrative interface, I reach the point where I have added the "class Admin: pass" to Poll and web browsed my way to see the first poll. Here, the poll is displayed, and the date description shows a broken character (the accented o): "Fecha de publicaci�n:" (not sure if that will get through, it shows a question mark inside a rotated square).
Well, I guess maybe Django wants unicode characters or stuff. So I modified my Poll class to read (note the unicode string):
pub_date = models.DateTimeField(u'Fecha de publicación')
Reloading the admin page I get:
UnicodeDecodeError at /admin/polls/poll/1/
'ascii' codec can't decode byte 0xc3 in position 75: ordinal not in range(128)
...along with a hell long of a callstack and environment dump. Since I'm used to these kind of things, I tried the following change:
pub_date = models.DateTimeField(u'Fecha de publicación'.encode("utf8"))
...which worked and didn't crash, but strikes me as unusually cumbersome to do. AFAICS the model doesn't properly treat unicode strings and the reason I get this OK with the utf8 encoding is just because that's what the output page charset is, and the result is processed by the browser. I guess if I switched that to latin1 now it would break visually as well, right? Is Django unable to handle internationalised stuff in the model?
If this is the case, I would add a note on the kind of strings you can use for the model.
Change History (5)
comment:1 by , 18 years ago
comment:2 by , 18 years ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
Looks like this isn't a bug. (See previous comment.)
comment:3 by , 18 years ago
Component: | Documentation → Internationalization |
---|---|
Resolution: | fixed |
Status: | closed → reopened |
I'm reopening this ticket because I'm experiencing the same issue and to post some additional information:
SVN revision 3065
first line of models.py has:
# -*- coding: iso-8859-1 -*-
a model:
class Project(models.Model): ... description = models.CharField('descripcion', blank=True, maxlength=255, db_column='DESCRIPTION') ... class Admin: fields = ((None, { 'fields': ('title', 'description', 'client', 'start_date', 'leader', 'status')} ), ) list_display=('title', 'client', 'status', 'leader')
Django conf/global_settings.py has
DEFAULT_CHARSET = 'utf-8'
Project settings.py has
DEBUG = True LANGUAGE_CODE = 'es-ar'
$ file models.py models.py: ASCII English text
If I edit the field verbose name to add a non-ASCII char
description = models.CharField('descripción', blank=True, maxlength=255, db_column='DESCRIPTION')
$ file models.py models.py: ISO-8859 English text
And then whe trying to access the edit page for that model by selecting it from the change list page of the admin app, I get a 'Descripci�n:' field label.
I'm using the bundled Django development Web server.
Server-side info:
Linux, python 2.3.5
Client-side info:
Mozilla Firefox 1.5.0.4 / Win32 / Windows 2000 + SP4 English
View -> Character Encoding -> UTF-8 is checked, if I change it to Western (ISO-8859-1) the
problem goes away.
When I change the verbose name to include the u prefix
description = models.CharField(u'descripción', blank=True, maxlength=255, db_column='DESCRIPTION')
I also get the exception + back trace debug page:
Request Method: GET Request URL: http://<my server DNS name>:8000/admin/intro/project/81/ Exception Type: UnicodeDecodeError Exception Value: 'ascii' codec can't decode byte 0xc3 in position 222: ordinal not in range(128) Exception Location: /usr/lib/python2.3/site-packages/django/template/__init__.py in render, line 686
Traceback (most recent call last): File "/usr/lib/python2.3/site-packages/django/template/__init__.py" in render_node 701. result = node.render(context) File "/usr/lib/python2.3/site-packages/django/template/defaulttags.py" in render 113. nodelist.append(node.render(context)) File "/usr/lib/python2.3/site-packages/django/template/defaulttags.py" in render 115. return nodelist.render(context) File "/usr/lib/python2.3/site-packages/django/template/__init__.py" in render 686. return ''.join(bits) UnicodeDecodeError at /admin/intro/project/81/ 'ascii' codec can't decode byte 0xc3 in position 222: ordinal not in range(128)
Why is the text getting decoded with the ascii codec?.
Feel free to ask for further info bacause I have the environment at hand.
Regards,
--
Ramiro
comment:4 by , 18 years ago
More info:
"Request information - META" section of the backtrace:
CONTENT_LENGTH | ||
CONTENT_TYPE | 'text/plain' | |
DJANGO_SETTINGS_MODULE | 'tang.settings' | |
GATEWAY_INTERFACE | 'CGI/1.1' | |
HOME | '/home/ramiro' | |
HTTP_ACCEPT | 'text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5' | |
HTTP_ACCEPT_CHARSET | 'ISO-8859-1,utf-8;q=0.7,*;q=0.7' | |
HTTP_ACCEPT_ENCODING | 'gzip,deflate' | |
HTTP_ACCEPT_LANGUAGE | 'en-us,en;q=0.7,es-ar;q=0.3' | |
HTTP_CONNECTION | 'keep-alive' | |
HTTP_COOKIE | 'sessionid=145006e2d5e41bb0ae6e740c5c3e4811; SITESERVER=ID=f9aa77d7c40371371daebd95787b0042' | |
HTTP_HOST | '<my Django server DNS name>:8000' | |
HTTP_IF_MODIFIED_SINCE | 'Fri, 02 Jun 2006 18:46:50 GMT' | |
HTTP_IF_NONE_MATCH | '1a18096f877784f328f0045a20f26e9a' | |
HTTP_KEEP_ALIVE | '300' | |
HTTP_REFERER | 'http://<my Django server DNS name>:8000/admin/intro/project/' | |
HTTP_USER_AGENT | 'Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4' | |
LANG | 'en_US' | |
LANGUAGE | 'en_AR:en_US:en_GB:en' | |
LOGNAME | 'ramiro' | |
PATH_INFO | '/admin/intro/project/81/' | |
PWD | '/home/ramiro/src/tang' | |
QUERYSTRING | ||
REMOTE_ADDR | '<my workstation IP address>' | |
REMOTE_HOST | '<my workstation DNS name>' | |
REQUEST_METHOD | 'GET' | |
RUN_MAIN | 'true' | |
SCRIPT_NAME | ||
SERVER_NAME | '<my Django server DNS name>' | |
SERVER_PORT | '8000' | |
SERVER_PROTOCOL | 'HTTP/1.1' | |
SERVER_SOFTWARE | 'WSGIServer/0.1 Python/2.3.5' | |
TZ | 'America/Cordoba' | |
USER | 'ramiro' | |
_ | './manage.py' | |
wsgi.errors | <open file '<stderr>', mode 'w' at 0xb7d890a0> | |
wsgi.file_wrapper | <class django.core.servers.basehttp.FileWrapper at 0xb7a9e7dc> | |
wsgi.input | <socket._fileobject object at 0xb7031c34> | |
wsgi.multiprocess | False | |
wsgi.multithread | True | |
wsgi.run_once | False | |
wsgi.url_scheme | 'http' | |
wsgi.version | (1, 0) |
comment:5 by , 18 years ago
Resolution: | → worksforme |
---|---|
Status: | reopened → closed |
I'm seeing now that a solution to this problem was posted in one of the lasts comments of ticket #170 (I've tested it and it works).
So I'm closing this ticket again. Perhaps I will open another targeted at the documentation component because it seems a somewhat common pitfall for non English users (see #1715, http://groups.google.com/group/django-users/browse_thread/thread/d536902362a36b9c/0d5b920adc628005#0d5b920adc628005). If the recommended solution is in fact the right one then maybe a note could be added to the models docs stating that for non ASCII languages it is beter to encode the models.py file in UTF-8 and flag it so by using magic comments as per PEP 0263
Regards,
I cannot reproduce this.
I used:
with a verbose name of:
and in my admin I see:
Did you check your browser to see if it selected/auto-detected the right page encoding? It should be UTF-8.
Note that I am using the latest SVN version and you didn't specify yours though.