Opened 11 years ago

Closed 11 years ago

#7412 closed (invalid)

i18n crash on non-ASCII (UTF-8 encoded) doctrings

Reported by: AV <av0000@…> Owned by: nobody
Component: contrib.admin Version: master
Severity: Keywords: i18n utf8 unicode
Cc: av0000@… Triage Stage: Unreviewed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


The Unicode decoding error occured in when source file contains non-ASCII docstrings.


Navigate to yoursite/documentation/models using following model example (the source file is in utf-8 encoding)

# -*- coding: utf-8 -*-
# ...
class TestModel(models.Model):
    Test description with non-ASCII symbols.

Workaround #1: use u"""....""" format but not on TAGS - tag parser is different :(

Workaround/patch #2: patch or change django/templatetags/ TranslateNode.render: from translation.ugettext(value) to translation.ugettext(smart_unicode(value))

Attachments (1)

i18n.diff (826 bytes) - added by AV <av0000@…> 11 years ago.

Download all attachments as: .zip

Change History (4)

Changed 11 years ago by AV <av0000@…>

Attachment: i18n.diff added

comment:1 Changed 11 years ago by Simon Greenhill

Triage Stage: UnreviewedReady for checkin

comment:2 Changed 11 years ago by Marc Garcia

Triage Stage: Ready for checkinUnreviewed

I can't reproduce the error. Adding non ascii characters to the docstrings on the model doesn't make django raise any encoding error. Could you provide more information about this issue please?

comment:3 Changed 11 years ago by Karen Tracey <kmtracey@…>

Resolution: invalid
Status: newclosed

I was able to recreate using the TestModel exactly as shown. However I don't believe that is a valid testcase. Per PEP 257 (

For Unicode docstrings, use u"""Unicode triple-quoted strings""".

That is, the model should be specified like so:

# -*- coding: utf-8 -*-
# ...
class TestModel(models.Model):
    Test description with non-ASCII symbols.

I believe that is the correct way to avoid the problem, since the model's __doc__ attribute will then be a Unicode object instead of a bytestring. Django can't really fix up a bytestring returned by __doc__ after the fact because the correct encoding is buried in the 'coding:' specification on the first line of the file containing the Model definition. The patch provided probably seemed to work because the system default locale encoding matched the file encoding, but you cannot count on that. I did verify that the recommended way of specifying the docstring works correctly on newforms-admin.

Note: See TracTickets for help on using tickets.
Back to Top