Opened 11 years ago

Closed 11 years ago

Last modified 11 years ago

#19606 closed Bug (fixed)

Oracle unicode insert fails.

Reported by: Jani Tiainen Owned by: nobody
Component: Database layer (models, ORM) Version: 1.4
Severity: Normal Keywords: oracle orm unicode
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


smart_str (which detection selects) does not work.

using force_unicode works (except for connection string).

Depending on what OCI client or instant client 11.2 is used when compiling cx_Oracle causes variation. doesn't work with smart_str while 11.2 does work.

Both can take plain unicode (u'<some unicode stuff here>') when using just cx_Oracle directly without any problems.


If I add manually some unicode to database Django can read it without any problems.

Change History (9)

comment:2 by Anssi Kääriäinen, 11 years ago

Triage Stage: UnreviewedAccepted

I haven't haven't been able to verify this (no here). The report seems well researched in the django-users thread so accepted based on that.

comment:3 by Jani Tiainen, 11 years ago

Problem is that Django makes incorrect detection what to string encoding system to use:

I did also some source code diving to make sure how this works:

In version 5.0.x cx_Oracle.UNICODE is only defined if code was compiled without flag WITH_UNICODE.

In version 5.1+ cx_Oracle.UNICODE is always defined if Python major version compiled against is < 3.

According to changelog of cx_Oracle:

Changes from 5.0.4 to 5.1

1) Remove support for UNICODE mode and permit Unicode to be passed through in

everywhere a string may be passed in. This means that strings will be
passed through to Oracle using the value of the NLS_LANG environment
variable in Python 3.x as well. Doing this eliminated a bunch of problems
that were discovered by using UNICODE mode and also removed an unnecessary
restriction in Python 2.x that Unicode could not be used in connect strings
or SQL statements, for example.

So behaviour should be: If cx_Oracle version is greater or equal to 5.1 real unicode should be used always. With pre-5.1 current behaviour is sufficient.

comment:4 by Anssi Kääriäinen <akaariai@…>, 11 years ago

Resolution: fixed
Status: newclosed

In d194f290571f7e9dda7d2fd7a6f2b171120f2f14:

Fixed #19606 -- Adjusted cx_Oracle unicode detection.

comment:5 by Anssi Kääriäinen <akaariai@…>, 11 years ago

In 457290326fa38297467922d827c630ae2026e88b:

[1.5.x] Fixed #19606 -- Adjusted cx_Oracle unicode detection.

Backpatch of d194f290571f7e9dda7d2fd7a6f2b171120f2f14

comment:6 by Anssi Kääriäinen, 11 years ago

I felt very uncomfortable about backporting this as I can't actually test the combo this is supposed to fix - that is, cx_oracle 5.1 + instantclient 10.2 as that wont compile on my machine. Based on the patch, cx_oracle 5.0 users should no get any change and I have tested 5.1 + instantclient 11.

I think this falls into the category of patches we can still backpatch to 1.5 at this stage - the combination of cx_oracle 5.1 + instantclient 10.2 is supported, but it doesn't work with unicode values currently.

So, after using too much time trying to tests this, I did backpatch this on the grounds that instanclient 11 works, cx_oracle 5.0 works as always, and the research done by jtiai shows that this is the right thing to do for 5.0 + instantclient 10.

comment:7 by Jani Tiainen, 11 years ago

According to my research it's all about version of cx_Oracle 5.0.x series does behave have slightly different definition for cx_Oracle.UNICODE than 5.1.x

It has nothing to do with OCI version linked against even I though that was the reason. But I think 1.5 is sufficient at this point.

comment:8 by Anssi Kääriäinen <akaariai@…>, 11 years ago

In 457290326fa38297467922d827c630ae2026e88b:

[1.5.x] Fixed #19606 -- Adjusted cx_Oracle unicode detection.

Backpatch of d194f290571f7e9dda7d2fd7a6f2b171120f2f14

comment:9 by anonymous, 11 years ago

#20292 is possibly related, at least reverting this change fixes that bug.

Note: See TracTickets for help on using tickets.
Back to Top