Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#19273 closed Bug (fixed)

PostgreSQL 9.0 cache db backend doesn't work on py3

Reported by: akaariai Owned by: nobody
Component: Core (Cache system) Version: 1.5-alpha-1
Severity: Release blocker Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

PostgreSQL + 9.0 + cache db backend doesn't work on python 3.

The reason for this is that base64 encoding a string returns bytes on py3, PostgreSQL 9.0 has hex output type conversion on by default for byte values, and the column we are inserting into is TEXT. So, psycopg2 gets a string "abc" and converts this to "hex encoding" ('\x616263'). However as the column is of type text this is inserted directly as string '\x616263' into the column. When we fetch it back we get a string containing '\x616263', not bytes containing "abc". This results in exception when trying to base64 decode the value.

On PostgreSQL versions earlier than 9.0 the default byte encoding style was 'encoded', which means we insert the "abc" as string 'abc' and everything works.

The solution is to insert the string "abc" into the DB, not bytes('abc'). Will post a patch soon...

Change History (3)

comment:1 Changed 3 years ago by akaariai

  • Has patch set

Here is the proposed patch: https://github.com/akaariai/django/commit/b07ae5ec84acaa1ecffc74ade51c4564632ca869

The 'cache' tests pass on all core backends on py2.7 and 3.2.

comment:2 Changed 3 years ago by Anssi Kääriäinen <akaariai@…>

  • Resolution set to fixed
  • Status changed from new to closed

In cc0ac26f4a3947be8a3fc55d4784d3474b640c23:

Fixed #19273 -- Fixed DB cache backend on pg 9.0+ and py3

There was a problem caused by Postgres 9.0+ having bytea_output default
value of 'hex' and cache backend inserting the content as 'bytes' into
a column of type TEXT. Fixed by converting the bytes value to a string
before insert.

comment:3 Changed 3 years ago by Anssi Kääriäinen <akaariai@…>

In 825a793555aafcbd670e50759f9a5046f11703df:

[1.5.x] Fixed #19273 -- Fixed DB cache backend on pg 9.0+ and py3

There was a problem caused by Postgres 9.0+ having bytea_output default
value of 'hex' and cache backend inserting the content as 'bytes' into
a column of type TEXT. Fixed by converting the bytes value to a string
before insert.

Backpatch of [cc0ac26f4a3947be8a3fc55d4784d3474b640c23].

Note: See TracTickets for help on using tickets.
Back to Top