Code

Opened 8 years ago

Closed 8 years ago

Last modified 7 years ago

#2514 closed defect (wontfix)

[patch] psycopg2 backend doesn't properly save/load unicode strings

Reported by: crankycoder@… Owned by: adrian
Component: contrib.admin Version:
Severity: normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

Django doesn't properly set the unicode encoding on the psycopg2 driver so when you save unicode strings into an attribute, you get raw strings back which you have to manually decode into proper unicode strings.

Here's what Django does now:

In [1]: from travel.models import *

In [2]: t = Trip.objects.get(pk=1)

In [3]: t.name
Out[3]: '\xc3\x90'

In [4]: t.name = u'\xd0'

In [5]: t.name
Out[5]: u'\xd0'

In [6]: t.save()

In [7]: t.name
Out[7]: u'\xd0'

In [8]: t = Trip.objects.get(pk=1)

In [9]: t.name
Out[9]: '\xc3\x90'

In [10]: t.name.decode("utf8")
Out[10]: u'\xd0'

Here's what it does after the patch:

In [1]: from travel.models import *

In [2]: t = Trip.objects.get(pk=1)

In [3]: t.name
Out[3]: u'Taste of Maple'

In [4]: t.name = u'\xd0'

In [5]: t.name
Out[5]: u'\xd0'

In [6]: t.save()

In [7]: t.name
Out[7]: u'\xd0'

In [8]: t = Trip.objects.get(pk=1)

In [9]: t.name
Out[9]: u'\xd0'

In [10]:

patch:

Index: base.py
===================================================================
--- base.py     (revision 3548)
+++ base.py     (working copy)
@@ -7,6 +7,9 @@
 from django.db.backends import util
 try:
     import psycopg2 as Database
+    #  Register unicode conversions
+    import psycopg2.extensions
+    psycopg2.extensions.register_type(psycopg2.extensions.UNICODE)
 except ImportError, e:
     from django.core.exceptions import ImproperlyConfigured
     raise ImproperlyConfigured, "Error loading psycopg2 module: %s" % e

Attachments (0)

Change History (5)

comment:1 Changed 8 years ago by anonymous

  • Summary changed from psycopg2 backend doesn't properly save/load unicode strings to [patch] psycopg2 backend doesn't properly save/load unicode strings

comment:2 Changed 8 years ago by adrian

  • Resolution set to fixed
  • Status changed from new to closed

(In [3565]) Fixed #2514 -- Improved psycopg2 backend to save/load Unicode strings correctly. Thanks, crankycoder@…

comment:3 Changed 8 years ago by Victor Ng <victor.ng@…>

  • Resolution fixed deleted
  • Status changed from closed to reopened

rolled back as of changeset [3675]

comment:4 Changed 8 years ago by Victor Ng <crankycoder@…>

  • Resolution set to wontfix
  • Status changed from reopened to closed

This change won't be put in until Django gets full unicode support. For now - either encode/decode to UTF8 when going to the database or call the psycopg2.extensions.register_type function in your own code.

See the django-dev thread for more details.

comment:5 Changed 7 years ago by Chris Wagner <cw264701@…>

FYI, it looks like psycopg2.extensions.register_type(psycopg2.extensions.UNICODE) is now called in the postgres_psycopg2 backend -- probably because the Unicode branch has been merged into trunk.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.