Opened 11 years ago

Closed 11 years ago

#20007 closed Bug (fixed)

psycopg2 should convert SQL arrays into python unicode-arrays

Reported by: hogbait@… Owned by: EricBoersma
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords: psycopg2 unicode python2
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: yes UI/UX: no

Description

Both test cases should succeed. However, the second one fails, because the cursor returns a utf-8 encoded string array instead of a unicode array.

#!/usr/bin/env python
# coding=utf8

from django.test import TestCase
#Assuming that the default connection uses psycopg2
from django.db import connection

class UnicodeArrayTestCase(TestCase):

    def select(self, val):
        cursor = connection.cursor()
        cursor.execute("select %s", (val,))
        return cursor.fetchone()[0]

    def test_select_ascii_array(self):
        a = ["awef"]
        b = self.select(a)
        self.assertEqual(a[0], b[0])

    def test_select_unicode_array(self):
        a = [u"ᄲawef"]
        b = self.select(a)
        self.assertEqual(a[0], b[0])

The django currently configures psycopg2 to convert strings to strings, but is not configured to convert string arrays into unicode arrays. A potential fix is described here: https://github.com/django/django/pull/189

However, you should probably put the change in the cursor constructor as described here: https://github.com/django/django/commit/6595ee896a9e3bf9cc1a9bd34af0e13bd32d8a3b

Psycopg2 documentation: http://initd.org/psycopg/docs/extensions.html#psycopg2.extensions.UNICODEARRAY

Change History (6)

comment:1 by Zack Drach, 11 years ago

Version: 1.5master

comment:2 by Aymeric Augustin, 11 years ago

Triage Stage: UnreviewedAccepted

I'm not familiar with Postgres arrays but this looks reasonable.

Last edited 11 years ago by Aymeric Augustin (previous) (diff)

comment:3 by Areski Belaid, 11 years ago

Can you add an example when you will have this issue with the Django ORM?

comment:4 by Tim Graham, 11 years ago

Patch needs improvement: set

The test needs to be incorporated into the Django test suite as well.

comment:5 by EricBoersma, 11 years ago

Owner: changed from nobody to EricBoersma
Status: newassigned

comment:6 by Tim Graham <timograham@…>, 11 years ago

Resolution: fixed
Status: assignedclosed

In ded40142a92813a8901faa4a6ca398e6679152f6:

Fixed #20007 -- Configured psycopg2 to return UnicodeArrays

Thanks hogbait for the report.

Note: See TracTickets for help on using tickets.
Back to Top