Opened 18 years ago

Closed 18 years ago

Last modified 17 years ago

#2514 closed defect (wontfix)

[patch] psycopg2 backend doesn't properly save/load unicode strings

Reported by: crankycoder@… Owned by: Adrian Holovaty
Component: contrib.admin Version:
Severity: normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Django doesn't properly set the unicode encoding on the psycopg2 driver so when you save unicode strings into an attribute, you get raw strings back which you have to manually decode into proper unicode strings.

Here's what Django does now:

In [1]: from travel.models import *

In [2]: t = Trip.objects.get(pk=1)

In [3]: t.name
Out[3]: '\xc3\x90'

In [4]: t.name = u'\xd0'

In [5]: t.name
Out[5]: u'\xd0'

In [6]: t.save()

In [7]: t.name
Out[7]: u'\xd0'

In [8]: t = Trip.objects.get(pk=1)

In [9]: t.name
Out[9]: '\xc3\x90'

In [10]: t.name.decode("utf8")
Out[10]: u'\xd0'

Here's what it does after the patch:

In [1]: from travel.models import *

In [2]: t = Trip.objects.get(pk=1)

In [3]: t.name
Out[3]: u'Taste of Maple'

In [4]: t.name = u'\xd0'

In [5]: t.name
Out[5]: u'\xd0'

In [6]: t.save()

In [7]: t.name
Out[7]: u'\xd0'

In [8]: t = Trip.objects.get(pk=1)

In [9]: t.name
Out[9]: u'\xd0'

In [10]:

patch:

Index: base.py
===================================================================
--- base.py     (revision 3548)
+++ base.py     (working copy)
@@ -7,6 +7,9 @@
 from django.db.backends import util
 try:
     import psycopg2 as Database
+    #  Register unicode conversions
+    import psycopg2.extensions
+    psycopg2.extensions.register_type(psycopg2.extensions.UNICODE)
 except ImportError, e:
     from django.core.exceptions import ImproperlyConfigured
     raise ImproperlyConfigured, "Error loading psycopg2 module: %s" % e

Change History (5)

comment:1 by anonymous, 18 years ago

Summary: psycopg2 backend doesn't properly save/load unicode strings[patch] psycopg2 backend doesn't properly save/load unicode strings

comment:2 by Adrian Holovaty, 18 years ago

Resolution: fixed
Status: newclosed

(In [3565]) Fixed #2514 -- Improved psycopg2 backend to save/load Unicode strings correctly. Thanks, crankycoder@…

comment:3 by Victor Ng <victor.ng@…>, 18 years ago

Resolution: fixed
Status: closedreopened

rolled back as of changeset [3675]

comment:4 by Victor Ng <crankycoder@…>, 18 years ago

Resolution: wontfix
Status: reopenedclosed

This change won't be put in until Django gets full unicode support. For now - either encode/decode to UTF8 when going to the database or call the psycopg2.extensions.register_type function in your own code.

See the django-dev thread for more details.

comment:5 by Chris Wagner <cw264701@…>, 17 years ago

FYI, it looks like psycopg2.extensions.register_type(psycopg2.extensions.UNICODE) is now called in the postgres_psycopg2 backend -- probably because the Unicode branch has been merged into trunk.

Note: See TracTickets for help on using tickets.
Back to Top