Code

Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#14417 closed (invalid)

UnicodeDecodeError exception in recent actions

Reported by: ctetruite Owned by: nobody
Component: contrib.admin Version: 1.2
Severity: Keywords: unicode
Cc: Triage Stage: Unreviewed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

Here is my situation.

I have a model with this verbose_name in meta data : u"Matériel d'occasion". The thing is this is a non ascii string. I do some stuff on an object of this model. Then, when I go the the admin page, I get this server error :

Caught UnicodeDecodeError while rendering: ('ascii', "Mat\xc3\xa9riel d'occasion", 3, 4, 'ordinal not in range(128)')

It occurs in the template django/contrib/admin/templates/admin/index.html at line 70 :

<span class="mini quiet">{% filter capfirst %}{% trans entry.content_type.name %}{% endfilter %}</span>

Attachments (3)

log.patch (753 bytes) - added by ctetruite 4 years ago.
log.diff (830 bytes) - added by ctetruite 4 years ago.
unicode_model_name_test.diff (1.0 KB) - added by claudep 4 years ago.
Add test for unicode model name

Download all attachments as: .zip

Change History (12)

Changed 4 years ago by ctetruite

comment:1 Changed 4 years ago by ctetruite

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

Stack trace

Traceback:
File "/usr/lib/python2.6/site-packages/django/core/handlers/base.py" in get_response
  100.                     response = callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python2.6/site-packages/django/contrib/admin/sites.py" in wrapper
  207.                 return self.admin_view(view, cacheable)(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/django/utils/decorators.py" in _wrapped_view
  76.                     response = view_func(request, *args, **kwargs)
File "/usr/lib/python2.6/site-packages/django/views/decorators/cache.py" in _wrapped_view_func
  69.         response = view_func(request, *args, **kwargs)
File "/usr/lib/python2.6/site-packages/django/contrib/admin/sites.py" in inner
  190.             return view(request, *args, **kwargs)
File "/usr/lib/python2.6/site-packages/django/views/decorators/cache.py" in _wrapped_view_func
  69.         response = view_func(request, *args, **kwargs)
File "/usr/lib/python2.6/site-packages/django/contrib/admin/sites.py" in index
  396.             context_instance=context_instance
File "/usr/lib/python2.6/site-packages/django/shortcuts/__init__.py" in render_to_response
  20.     return HttpResponse(loader.render_to_string(*args, **kwargs), **httpresponse_kwargs)
File "/usr/lib/python2.6/site-packages/django/template/loader.py" in render_to_string
  186.     return t.render(context_instance)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in render
  173.             return self._render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in _render
  167.         return self.nodelist.render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in render
  796.                 bits.append(self.render_node(node, context))
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  72.             result = node.render(context)
File "/usr/lib/python2.6/site-packages/django/template/loader_tags.py" in render
  125.         return compiled_parent._render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in _render
  167.         return self.nodelist.render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in render
  796.                 bits.append(self.render_node(node, context))
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  72.             result = node.render(context)
File "/usr/lib/python2.6/site-packages/django/template/loader_tags.py" in render
  125.         return compiled_parent._render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in _render
  167.         return self.nodelist.render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in render
  796.                 bits.append(self.render_node(node, context))
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  72.             result = node.render(context)
File "/usr/lib/python2.6/site-packages/django/template/loader_tags.py" in render
  62.             result = block.nodelist.render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in render
  796.                 bits.append(self.render_node(node, context))
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  72.             result = node.render(context)
File "/usr/lib/python2.6/site-packages/django/template/defaulttags.py" in render
  260.             return self.nodelist_false.render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in render
  796.                 bits.append(self.render_node(node, context))
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  72.             result = node.render(context)
File "/usr/lib/python2.6/site-packages/django/template/defaulttags.py" in render
  174.                 nodelist.append(node.render(context))
File "/usr/lib/python2.6/site-packages/django/template/defaulttags.py" in render
  258.             return self.nodelist_true.render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in render
  796.                 bits.append(self.render_node(node, context))
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  72.             result = node.render(context)
File "/usr/lib/python2.6/site-packages/django/template/defaulttags.py" in render
  82.         output = self.nodelist.render(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in render
  796.                 bits.append(self.render_node(node, context))
File "/usr/lib/python2.6/site-packages/django/template/debug.py" in render_node
  72.             result = node.render(context)
File "/usr/lib/python2.6/site-packages/django/templatetags/i18n.py" in render
  45.         output = self.filter_expression.resolve(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in resolve
  553.                 obj = self.var.resolve(context)
File "/usr/lib/python2.6/site-packages/django/template/__init__.py" in resolve
  701.             return _(value)
File "/usr/lib/python2.6/site-packages/django/utils/translation/__init__.py" in ugettext
  55.     return real_ugettext(message)
File "/usr/lib/python2.6/site-packages/django/utils/functional.py" in _curried
  55.         return _curried_func(*(args+moreargs), **dict(kwargs, **morekwargs))
File "/usr/lib/python2.6/site-packages/django/utils/translation/__init__.py" in delayed_loader
  36.     return getattr(trans, real_name)(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/django/utils/translation/trans_real.py" in ugettext
  276.     return do_translate(message, 'ugettext')
File "/usr/lib/python2.6/site-packages/django/utils/translation/trans_real.py" in do_translate
  262.         result = getattr(t, translation_function)(eol_message)
File "/usr/lib/python2.6/gettext.py" in ugettext
  404.             return unicode(message)

Exception Type: TemplateSyntaxError at /admin/
Exception Value: Caught UnicodeDecodeError while rendering: ('ascii', "Mat\xc3\xa9riel d'occasion", 3, 4, 'ordinal not in range(128)')

Changed 4 years ago by ctetruite

Changed 4 years ago by claudep

Add test for unicode model name

comment:2 Changed 4 years ago by claudep

I've just added a test that specifically tests this. But on my system, it passes without your patch.

Why would entry.content_type.name not be an Unicode string in the first place?

The following snippet could indicate you if ContentType names are really all Unicode strings.

from django.contrib.contenttypes.models import ContentType
for c in ContentType.objects.all():
    print c.name.__class__, c.name

comment:3 Changed 4 years ago by ctetruite

So I've learnt how to use the test suite. Here are my results. Adding your unit test, it also passes for me. It outputs that when displaying content types: <type 'unicode'> ¿Chapter?

Then, I changed the Chapter verbose name from '¿Chapter?' to u"Matériel d'occasion" with a french accent. And so on in the unit test. Executing tests give me this:

<type 'unicode'> Matériel d'occasion

But the test fails with this error:

AssertionError: Couldn't find '<span class="mini quiet">Matériel d'occasion</span>' in response

Did I configured models in the wrong way ?

comment:4 Changed 4 years ago by claudep

That's a totally different error. Simple quotes are converted to &#39; in the HTML result. So your test case should test for:

self.assertContains(response, """<span class="mini quiet">Matériel d&#39;occasion</span>""")

comment:5 Changed 4 years ago by ctetruite

Oops, sorry. No problem with the test suite: sqlite or mysql.

With my application in dev mode, using MySql (DEFAULT CHARACTER SET utf8 COLLATE utf8_bin), I tried accessing admin page. I've just printed content_type class value at the same place that in my patch. I get this:

Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
<type 'str'> Matériel d'occasion
<type 'str'> Matériel d'occasion
<type 'str'> Matériel d'occasion
[02/Dec/2010 21:15:48] "GET /admin/ HTTP/1.1" 500 394166

Whereas in the test suite it's <unicode>. In the database:

select * from django_content_type;
Matériel d'occasion | website      | occmaterial |
...

comment:6 Changed 4 years ago by claudep

I suspect some database configuration problem. Did you try to get the class value of any other CharField on one of your own models?

comment:7 Changed 4 years ago by claudep

  • Resolution set to invalid
  • Status changed from new to closed

Well, ramiro friendly pointed me to http://docs.djangoproject.com/en/1.2/ref/databases/#collation-settings
It is surely the cause of your problems (using utf8_bin collation).

comment:8 Changed 4 years ago by ramiro

(In [14779]) Fixed grammar and tweaked notes about MySQL database/table collation interaction with text fields. Refs #14417.

comment:9 Changed 4 years ago by ramiro

(In [14780]) [1.2.X] Fixed grammar and tweaked notes about MySQL database/table collation interaction with text fields. Refs #14417.

Backport of [14779] from trunk

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.