Opened 4 years ago

Closed 3 years ago

#20007 closed Bug (fixed)

psycopg2 should convert SQL arrays into python unicode-arrays

Reported by: hogbait@… Owned by: EricBoersma
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords: psycopg2 unicode python2
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: yes UI/UX: no


Both test cases should succeed. However, the second one fails, because the cursor returns a utf-8 encoded string array instead of a unicode array.

#!/usr/bin/env python
# coding=utf8

from django.test import TestCase
#Assuming that the default connection uses psycopg2
from django.db import connection

class UnicodeArrayTestCase(TestCase):

    def select(self, val):
        cursor = connection.cursor()
        cursor.execute("select %s", (val,))
        return cursor.fetchone()[0]

    def test_select_ascii_array(self):
        a = ["awef"]
        b =
        self.assertEqual(a[0], b[0])

    def test_select_unicode_array(self):
        a = [u"ᄲawef"]
        b =
        self.assertEqual(a[0], b[0])

The django currently configures psycopg2 to convert strings to strings, but is not configured to convert string arrays into unicode arrays. A potential fix is described here:

However, you should probably put the change in the cursor constructor as described here:

Psycopg2 documentation:

Change History (6)

comment:1 Changed 4 years ago by Zack Drach

Version: 1.5master

comment:2 Changed 4 years ago by Aymeric Augustin

Triage Stage: UnreviewedAccepted

I'm not familiar with Postgres arrays but this looks reasonnable.

Version 0, edited 4 years ago by Aymeric Augustin (next)

comment:3 Changed 3 years ago by Areski Belaid

Can you add an example when you will have this issue with the Django ORM?

comment:4 Changed 3 years ago by Tim Graham

Patch needs improvement: set

The test needs to be incorporated into the Django test suite as well.

comment:5 Changed 3 years ago by EricBoersma

Owner: changed from nobody to EricBoersma
Status: newassigned

comment:6 Changed 3 years ago by Tim Graham <timograham@…>

Resolution: fixed
Status: assignedclosed

In ded40142a92813a8901faa4a6ca398e6679152f6:

Fixed #20007 -- Configured psycopg2 to return UnicodeArrays

Thanks hogbait for the report.

Note: See TracTickets for help on using tickets.
Back to Top