Opened 9 years ago

Last modified 2 months ago

#6517 new Bug

MySQL: manage.py dbshell does not get charset from DATABASES setting

Reported by: Tom Vergote Owned by:
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords:
Cc: julie@…, victoria@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I noticed that manage.py dbshell doesn't respect the database_options.
I ran into an issue with an application we are creating that needs to support mysql and postgre at least, we execute some sql scripts that get piped to manage.py dbshell (to avoid hardcoding psql -U xxx or mysql -u xxx and creating 2 scripts every time).

When running an utf8 database with utf8 as our charset in database_options, we ran into some weird encoding issues.

The solution for us was to learn mysql/client.py to respect the encoding settings in settings.py

Are you opposed to something like this?

Attaching small patch that fixes our problem. Let me know if it needs extending to support other backends or database_options.

Attachments (1)

mysql_client.patch (776 bytes) - added by Tom Vergote 9 years ago.
small patch to illustrate suggestion

Download all attachments as: .zip

Change History (12)

Changed 9 years ago by Tom Vergote

Attachment: mysql_client.patch added

small patch to illustrate suggestion

comment:1 Changed 9 years ago by Tom Vergote

Has patch: set

Also please note that the read_default_file setting is respected, so it's somehow inconsistent behaviour

comment:2 Changed 9 years ago by Tom Vergote

Summary: manage.py dbshell doesn't respect database_optionsmanage.py dbshell does not get charset from database_options

comment:3 Changed 9 years ago by Simon Greenhill <dev@…>

Triage Stage: UnreviewedDesign decision needed

comment:4 Changed 6 years ago by Julien Phalip

Needs tests: set
Severity: Normal
Type: Bug

comment:5 Changed 5 years ago by Carl Meyer

Easy pickings: unset
Triage Stage: Design decision neededAccepted
UI/UX: unset

We should look into whether this is needed on other backends as well, not just MySQL.

comment:6 Changed 4 years ago by Julie Pichon

Cc: julie@… added

I adapted tvrg's patch to the current codebase - https://github.com/jpichon/django/compare/ticket_6517

I also tested dbshell with the sqlite3 and postgresql backends, and it works with UTF-8 characters without patching.

There are other ways to get the MySQL backend to play nicely with UTF-8 that don't require changing the code. You can add "default-character-set = utf8" to /etc/mysql/my.cnf on the server side, or create an option file with the UTF-8 charset config and update the settings to point to it in the "read_default_file" database options (as seen on https://docs.djangoproject.com/en/dev/ref/databases/#connecting-to-the-database)

[client]
default-character-set = utf8

If the patch option is preferred, I'm happy to work on the tests if someone could let me know where the dbshell tests live.

comment:7 Changed 4 years ago by Julie Pichon

Triage Stage: AcceptedDesign decision needed

comment:8 Changed 4 years ago by Anssi Kääriäinen

Triage Stage: Design decision neededAccepted

Seems like a good idea to me.

comment:9 Changed 17 months ago by Victoria Martínez de la Cruz

Cc: victoria@… added
Owner: changed from nobody to Victoria Martínez de la Cruz
Status: newassigned

comment:10 Changed 16 months ago by Tim Graham

Summary: manage.py dbshell does not get charset from database_optionsMySQL: manage.py dbshell does not get charset from DATABASES setting

A tests can be added in tests/dbshell/test_mysql.py.

comment:11 Changed 2 months ago by Tim Graham

Owner: Victoria Martínez de la Cruz deleted
Status: assignednew
Note: See TracTickets for help on using tickets.
Back to Top