#28067 closed Cleanup/optimization (fixed)
Clarify __str__() return type when using python_2_unicode_compatible()
Reported by: | Christophe Pettus | Owned by: | nobody |
---|---|---|---|
Component: | Documentation | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Accepted | |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
The str method (as implemented using six) strips off the "unicodeness" of a string. If chained together, this can result in an error:
from django.db import models from django.utils.encoding import python_2_unicode_compatible @python_2_unicode_compatible class A(models.Model): c = models.CharField(max_length=20) def __str__(self): return self.c @python_2_unicode_compatible class B(models.Model): a = models.ForeignKey(A) def __str__(self): return str(self.a)
>>> from test.models import A, B >>> a = A(c=u'réparer') >>> a.save() >>> b = B(a=a) >>> b.save() >>> a <A: réparer> >>> b <B: [Bad Unicode data]> >>> print b Traceback (most recent call last): File "<console>", line 1, in <module> File "/Users/xof/Documents/Dev/environments/peep/lib/python2.7/site-packages/django/utils/six.py", line 842, in <lambda> klass.__str__ = lambda self: self.__unicode__().encode('utf-8') UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
Note that the same behavior exists without @python_2_unicode_compatible, but the example uses so as to follow the correct path.
This is probably a bug in six (shouldn't str preserve its unicode type in this case?), but we might want to ameliorate it.
Change History (9)
comment:1 by , 7 years ago
Resolution: | → invalid |
---|---|
Status: | new → closed |
comment:2 by , 7 years ago
Then I'm confused in this case how to write code that runs on both Python 2 and Python 3 where you are retrieving text from the database. It comes back as type unicode (on Python 2); how should this be handled in a both 2 and 3 way?
comment:3 by , 7 years ago
As documnted you must return text and not bytes from __str__()
when using @python_2_unicode_compatible
.
from django.utils import six @python_2_unicode_compatible class B(models.Model): a = models.ForeignKey(A) def __str__(self): return six.text_type(self.a)
On Python 2 six.text_type is unicode
while six.text_type is str
on Python 3.
comment:4 by , 7 years ago
Then I think we have a documentation bug, because the docs don't actually say that; they just say: "To support Python 2 and 3 with a single code base, define a __str__
method returning text and apply this decorator to the class," and don't discuss six.text_type() at all (at least, not in the linked-to section).
comment:5 by , 7 years ago
Component: | Database layer (models, ORM) → Documentation |
---|---|
Resolution: | invalid |
Status: | closed → new |
Triage Stage: | Unreviewed → Accepted |
Type: | Bug → Cleanup/optimization |
Version: | 1.11 → master |
I think the documentation assumes the reader knows that returning text means returning unicode
on Python 2 and str
on Python 3. Wouldn't hurt to adjust the example to disambiguate that as basing __str__
methods output on related objects __str__
output is commonly used pattern.
comment:6 by , 7 years ago
It seems the most applicable advice is, "If you are using @python_2_unicode_compatible
to support both Python 2 and Python 3, cast objects using six.text_type
rather than str
or unicode
." Does that cover it?
comment:7 by , 7 years ago
Has patch: | set |
---|---|
Summary: | Encoding error when __str__ returns non-ASCII → Clarify __str__() return type when using python_2_unicode_compatible() |
When using
@python_2_unicode_compatible
,__str__()
must return "text" (unicode). Usingstr()
returns bytes on Python 2. Usesix.text_type(self.a)
instead.