Opened 7 years ago

Closed 7 years ago

#27219 closed Bug (fixed)

Emoji don't work in TextFields using Oracle backend

Reported by: dmedvinsky Owned by: nobody
Component: Database layer (models, ORM) Version: 1.10
Severity: Normal Keywords: oracle, unicode
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

When using cx_Oracle as a backend, I can't use Emoji in TextFields, which are stored as NCLOBs in the database. When inputting an emoji and saving them in the database, upon retrieving I get four replacement charaters (�) per emoji instead.

Django ​uses hardcoded NLS_LANG=".UTF8" variable, which is outdated, only supports Unicode standard version 3 and cannot properly store 4-byte characters. ​The Oracle documentation recommends using the AL32UTF8 encoding instead, which supports latest Unicode standard and stores 4-byte characters properly.

The change that fixes the emoji for me: ​https://github.com/django/django/pull/7241

I recognize that it needs some tests, so I seek the help from those with better knowledge of Django test cases than me. If someone can kindly direct me, I can create the test for this case.

Change History (6)

comment:1 Changed 7 years ago by dmedvinsky

Needs tests: set

comment:2 Changed 7 years ago by Tim Graham

Triage Stage: Unreviewed → Accepted

comment:3 Changed 7 years ago by dmedvinsky

Needs tests: unset

comment:4 Changed 7 years ago by Tim Graham

Patch needs improvement: set

The new tests don't pass on MySQL due to #18392. We can either fix that issue first, or skip the test on MySQL.

comment:5 Changed 7 years ago by Tim Graham

Patch needs improvement: unset

comment:6 Changed 7 years ago by Tim Graham <timograham@…>

Resolution: → fixed
Status: new → closed

In 1a9f6db5:

Fixed #27219 -- Changed cx_Oracle client encoding to AL32UTF8 to allow 4-byte characters.

Note: See TracTickets for help on using tickets.
Back to Top