Opened 7 years ago

Closed 7 years ago

#27219 closed Bug (fixed)

Emoji don't work in TextFields using Oracle backend

Reported by: dmedvinsky Owned by: nobody
Component: Database layer (models, ORM) Version: 1.10
Severity: Normal Keywords: oracle, unicode
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


When using cx_Oracle as a backend, I can't use Emoji in TextFields, which are stored as NCLOBs in the database. When inputting an emoji and saving them in the database, upon retrieving I get four replacement charaters (�) per emoji instead.

Django uses hardcoded NLS_LANG=".UTF8" variable, which is outdated, only supports Unicode standard version 3 and cannot properly store 4-byte characters. The Oracle documentation recommends using the AL32UTF8 encoding instead, which supports latest Unicode standard and stores 4-byte characters properly.

The change that fixes the emoji for me:

I recognize that it needs some tests, so I seek the help from those with better knowledge of Django test cases than me. If someone can kindly direct me, I can create the test for this case.

Change History (6)

comment:1 by dmedvinsky, 7 years ago

Needs tests: set

comment:2 by Tim Graham, 7 years ago

Triage Stage: UnreviewedAccepted

comment:3 by dmedvinsky, 7 years ago

Needs tests: unset

comment:4 by Tim Graham, 7 years ago

Patch needs improvement: set

The new tests don't pass on MySQL due to #18392. We can either fix that issue first, or skip the test on MySQL.

comment:5 by Tim Graham, 7 years ago

Patch needs improvement: unset

comment:6 by Tim Graham <timograham@…>, 7 years ago

Resolution: fixed
Status: newclosed

In 1a9f6db5:

Fixed #27219 -- Changed cx_Oracle client encoding to AL32UTF8 to allow 4-byte characters.

Note: See TracTickets for help on using tickets.
Back to Top