Opened 8 years ago

Closed 8 years ago

#25875 closed Bug (fixed)

Unicode error when converting a Q object to a string

Reported by: Ben Kraft Owned by: Ben Kraft
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

If you have a model whose __str__ returns non-ASCII text, and you refer to that model in a Q object, attempting to print that Q object, or otherwise convert it to a string, will error. This is a regression in Django 1.5, caused indirectly by 3dd5d72, and remains an issue in master; I believe it should only affect python 2 but I haven't actually checked.

The issue appears to be that django.utils.tree.Node.__str__ inadvertently attempts to convert the string back to unicode. In particular, since 3dd5d72, self.connector is a unicode string if self is a Q object. This causes python to convert the string being formatted and all its formatting arguments to unicode. Since this is done implicitly it uses the default encoding (ASCII), which fails.

Fixing this should be as simple as just explicitly doing str(self.connector) instead, although I haven't yet run the tests to see if that breaks anything else. In any case, I'm happy to write a patch to do that later this week. It's possible it would be better to instead define a __unicode__ method and avoid the string-conversion entirely, I'm not sure, but this seems simpler.

Complete repro:

# in models.py
class Foo(models.Model):
    name = models.TextField()
    def __unicode__(self):
        return u'Foö'
    def __str__(self):
        return unicode(self).encode('utf-8')
    def __repr__(self):
        return str(self)

# in a shell
Q(baz=Foo(name="baz")) # raises UnicodeDecodeError

Note that the repro works as long as at least one of __unicode__, __str__, and __repr__ is defined and returns non-ASCII text -- they don't need to all be explicitly defined.

Change History (4)

comment:1 by Josh Smeaton, 8 years ago

Triage Stage: UnreviewedAccepted

comment:2 by Claude Paroz, 8 years ago

Has patch: set

comment:3 by Simon Charette, 8 years ago

Triage Stage: AcceptedReady for checkin

comment:4 by Claude Paroz <claude@…>, 8 years ago

Resolution: fixed
Status: newclosed

In ed20dd2e:

Fixed #25875 -- Prevented UnicodeDecodeError for Q object repr

Thanks Ben Kraft for the report, and Simon Charette for the review.

Note: See TracTickets for help on using tickets.
Back to Top