Opened 7 years ago

Last modified 2 years ago

#6517 new Bug

manage.py dbshell does not get charset from database_options

Reported by: tvrg Owned by: nobody
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords:
Cc: julie@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I noticed that manage.py dbshell doesn't respect the database_options.
I ran into an issue with an application we are creating that needs to support mysql and postgre at least, we execute some sql scripts that get piped to manage.py dbshell (to avoid hardcoding psql -U xxx or mysql -u xxx and creating 2 scripts every time).

When running an utf8 database with utf8 as our charset in database_options, we ran into some weird encoding issues.

The solution for us was to learn mysql/client.py to respect the encoding settings in settings.py

Are you opposed to something like this?

Attaching small patch that fixes our problem. Let me know if it needs extending to support other backends or database_options.

Attachments (1)

mysql_client.patch (776 bytes) - added by tvrg 7 years ago.
small patch to illustrate suggestion

Download all attachments as: .zip

Change History (9)

Changed 7 years ago by tvrg

small patch to illustrate suggestion

comment:1 Changed 7 years ago by tvrg

  • Has patch set
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

Also please note that the read_default_file setting is respected, so it's somehow inconsistent behaviour

comment:2 Changed 7 years ago by tvrg

  • Summary changed from manage.py dbshell doesn't respect database_options to manage.py dbshell does not get charset from database_options

comment:3 Changed 7 years ago by Simon Greenhill <dev@…>

  • Triage Stage changed from Unreviewed to Design decision needed

comment:4 Changed 4 years ago by julien

  • Needs tests set
  • Severity set to Normal
  • Type set to Bug

comment:5 Changed 3 years ago by carljm

  • Easy pickings unset
  • Triage Stage changed from Design decision needed to Accepted
  • UI/UX unset

We should look into whether this is needed on other backends as well, not just MySQL.

comment:6 Changed 3 years ago by jpichon

  • Cc julie@… added

I adapted tvrg's patch to the current codebase - https://github.com/jpichon/django/compare/ticket_6517

I also tested dbshell with the sqlite3 and postgresql backends, and it works with UTF-8 characters without patching.

There are other ways to get the MySQL backend to play nicely with UTF-8 that don't require changing the code. You can add "default-character-set = utf8" to /etc/mysql/my.cnf on the server side, or create an option file with the UTF-8 charset config and update the settings to point to it in the "read_default_file" database options (as seen on https://docs.djangoproject.com/en/dev/ref/databases/#connecting-to-the-database)

[client]
default-character-set = utf8

If the patch option is preferred, I'm happy to work on the tests if someone could let me know where the dbshell tests live.

comment:7 Changed 3 years ago by jpichon

  • Triage Stage changed from Accepted to Design decision needed

comment:8 Changed 2 years ago by akaariai

  • Triage Stage changed from Design decision needed to Accepted

Seems like a good idea to me.

Note: See TracTickets for help on using tickets.
Back to Top