Opened 13 years ago

Closed 13 years ago

Last modified 13 years ago

#14417 closed (invalid)

UnicodeDecodeError exception in recent actions

Reported by: ctetruite Owned by: nobody
Component: contrib.admin Version: 1.2
Severity: Keywords: unicode
Cc: Triage Stage: Unreviewed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Here is my situation.

I have a model with this verbose_name in meta data : u"Matériel d'occasion". The thing is this is a non ascii string. I do some stuff on an object of this model. Then, when I go the the admin page, I get this server error :

Caught UnicodeDecodeError while rendering: ('ascii', "Mat\xc3\xa9riel d'occasion", 3, 4, 'ordinal not in range(128)')

It occurs in the template django/contrib/admin/templates/admin/index.html at line 70 :

<span class="mini quiet">{% filter capfirst %}{% trans entry.content_type.name %}{% endfilter %}</span>

Attachments (3)

log.patch (753 bytes ) - added by ctetruite 13 years ago.
log.diff (830 bytes ) - added by ctetruite 13 years ago.
unicode_model_name_test.diff (1.0 KB ) - added by Claude Paroz 13 years ago.
Add test for unicode model name

Download all attachments as: .zip

Change History (12)

by ctetruite, 13 years ago

Attachment: log.patch added

comment:1 by ctetruite, 13 years ago

Stack trace

Traceback:
File "/usr/lib/python2.6/site-packages/django/core/handlers/base.py" in get_response
  100.                     response = callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python2.6/site-packages/django/contrib/admin/sites.py" in wrapper
  207.                 return self.admin_view(view, cacheable)(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/django/utils/decorators.py" in _wrapped_view
  76.                     response = view_func(request, *args, **kwargs)
File "/usr/lib/python2.6/site-packages/django/views/decorators/cache.py" in _wrapped_view_func
  69.         response = view_func(request, *args, **kwargs)
File "/usr/lib/python2.6/site-packages/django/contrib/admin/sites.py" in inner
  190.             return view(request, *args, **kwargs)
File "/usr/lib/python2.6/site-packages/django/views/decorators/cache.py" in _wrapped_view_func
  69.         response = view_func(request, *args, **kwargs)
File "/usr/lib/python2.6/site-packages/django/contrib/admin/sites.py" in index
  396.             context_instance=context_instance
File "/usr/lib/python2.6/site-packages/django/shortcuts/__init__.py" in render_to_response
  20.     return HttpResponse(loader.render_to_string(*args, **kwargs), **httpresponse_kwargs)
File "/usr/lib/python2.6/site-packages/django/template/loader.py" in render_to_string
  186.     return t.render(context_instance)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in render
  173.             return self._render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in _render
  167.         return self.nodelist.render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in render
  796.                 bits.append(self.render_node(node, context))
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  72.             result = node.render(context)
File "/usr/lib/python2.6/site-packages/django/template/loader_tags.py" in render
  125.         return compiled_parent._render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in _render
  167.         return self.nodelist.render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in render
  796.                 bits.append(self.render_node(node, context))
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  72.             result = node.render(context)
File "/usr/lib/python2.6/site-packages/django/template/loader_tags.py" in render
  125.         return compiled_parent._render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in _render
  167.         return self.nodelist.render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in render
  796.                 bits.append(self.render_node(node, context))
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  72.             result = node.render(context)
File "/usr/lib/python2.6/site-packages/django/template/loader_tags.py" in render
  62.             result = block.nodelist.render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in render
  796.                 bits.append(self.render_node(node, context))
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  72.             result = node.render(context)
File "/usr/lib/python2.6/site-packages/django/template/defaulttags.py" in render
  260.             return self.nodelist_false.render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in render
  796.                 bits.append(self.render_node(node, context))
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  72.             result = node.render(context)
File "/usr/lib/python2.6/site-packages/django/template/defaulttags.py" in render
  174.                 nodelist.append(node.render(context))
File "/usr/lib/python2.6/site-packages/django/template/defaulttags.py" in render
  258.             return self.nodelist_true.render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in render
  796.                 bits.append(self.render_node(node, context))
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  72.             result = node.render(context)
File "/usr/lib/python2.6/site-packages/django/template/defaulttags.py" in render
  82.         output = self.nodelist.render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in render
  796.                 bits.append(self.render_node(node, context))
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  72.             result = node.render(context)
File "/usr/lib/python2.6/site-packages/django/templatetags/i18n.py" in render
  45.         output = self.filter_expression.resolve(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in resolve
  553.                 obj = self.var.resolve(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in resolve
  701.             return _(value)
File "/usr/lib/python2.6/site-packages/django/utils/translation/__init__.py" in ugettext
  55.     return real_ugettext(message)
File "/usr/lib/python2.6/site-packages/django/utils/functional.py" in _curried
  55.         return _curried_func(*(args+moreargs), **dict(kwargs, **morekwargs))
File "/usr/lib/python2.6/site-packages/django/utils/translation/__init__.py" in delayed_loader
  36.     return getattr(trans, real_name)(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/django/utils/translation/trans_real.py" in ugettext
  276.     return do_translate(message, 'ugettext')
File "/usr/lib/python2.6/site-packages/django/utils/translation/trans_real.py" in do_translate
  262.         result = getattr(t, translation_function)(eol_message)
File "/usr/lib/python2.6/gettext.py" in ugettext
  404.             return unicode(message)

Exception Type: TemplateSyntaxError at /admin/
Exception Value: Caught UnicodeDecodeError while rendering: ('ascii', "Mat\xc3\xa9riel d'occasion", 3, 4, 'ordinal not in range(128)')

by ctetruite, 13 years ago

Attachment: log.diff added

by Claude Paroz, 13 years ago

Add test for unicode model name

comment:2 by Claude Paroz, 13 years ago

I've just added a test that specifically tests this. But on my system, it passes without your patch.

Why would entry.content_type.name not be an Unicode string in the first place?

The following snippet could indicate you if ContentType names are really all Unicode strings.

from django.contrib.contenttypes.models import ContentType
for c in ContentType.objects.all():
    print c.name.__class__, c.name

comment:3 by ctetruite, 13 years ago

So I've learnt how to use the test suite. Here are my results. Adding your unit test, it also passes for me. It outputs that when displaying content types: <type 'unicode'> ¿Chapter?

Then, I changed the Chapter verbose name from '¿Chapter?' to u"Matériel d'occasion" with a french accent. And so on in the unit test. Executing tests give me this:

<type 'unicode'> Matériel d'occasion

But the test fails with this error:

AssertionError: Couldn't find '<span class="mini quiet">Matériel d'occasion</span>' in response

Did I configured models in the wrong way ?

comment:4 by Claude Paroz, 13 years ago

That's a totally different error. Simple quotes are converted to &#39; in the HTML result. So your test case should test for:

self.assertContains(response, """<span class="mini quiet">Matériel d&#39;occasion</span>""")

comment:5 by ctetruite, 13 years ago

Oops, sorry. No problem with the test suite: sqlite or mysql.

With my application in dev mode, using MySql (DEFAULT CHARACTER SET utf8 COLLATE utf8_bin), I tried accessing admin page. I've just printed content_type class value at the same place that in my patch. I get this:

Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
<type 'str'> Matériel d'occasion
<type 'str'> Matériel d'occasion
<type 'str'> Matériel d'occasion
[02/Dec/2010 21:15:48] "GET /admin/ HTTP/1.1" 500 394166

Whereas in the test suite it's <unicode>. In the database:

select * from django_content_type;
Matériel d'occasion | website      | occmaterial |
...

comment:6 by Claude Paroz, 13 years ago

I suspect some database configuration problem. Did you try to get the class value of any other CharField on one of your own models?

comment:7 by Claude Paroz, 13 years ago

Resolution: invalid
Status: newclosed

Well, ramiro friendly pointed me to http://docs.djangoproject.com/en/1.2/ref/databases/#collation-settings
It is surely the cause of your problems (using utf8_bin collation).

comment:8 by Ramiro Morales, 13 years ago

(In [14779]) Fixed grammar and tweaked notes about MySQL database/table collation interaction with text fields. Refs #14417.

comment:9 by Ramiro Morales, 13 years ago

(In [14780]) [1.2.X] Fixed grammar and tweaked notes about MySQL database/table collation interaction with text fields. Refs #14417.

Backport of [14779] from trunk

Note: See TracTickets for help on using tickets.
Back to Top