Opened 9 years ago

Last modified 14 months ago

#5745 new New feature

MySQL Collations/Charsets and Engines

Reported by: Armin Ronacher Owned by: nobody
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords: mysql
Cc: lidaobing@…, jdchrist@…, cmawebsite@…, Shai Berger, me@… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Django should really have a DATABASE_CHARSET / DATABASE_COLLATION setting in the settings.py as well as a setting for the mysql database engine. It's stunning that unittests can enforce a database collation but tables not. Also an argument in the Meta section of models that allows to define the collation, charset and engine would be an important thing to have.

I know that there are workarounds for those limitations but they have all disadvantages.

Change History (16)

comment:1 Changed 9 years ago by Simon G <dev@…>

Triage Stage: UnreviewedDesign decision needed

comment:2 Changed 9 years ago by Dmitry Gorbunow

-0 to this. I think that Unicode should be used everythere by default.

comment:3 Changed 9 years ago by Armin Ronacher

Unicode yes. But collations do not affect unicode but the way it deals with unicode data (german sorting, french sorting, greek sorting etc.). And other encodings in the database are required if you deal with old database data. And currently django creates latin1 tables which is the database default.

comment:4 Changed 9 years ago by Dmitry Gorbunow

Anyways this is a database-specific thing.

comment:5 Changed 9 years ago by Armin Ronacher

So is the Oracle db_tablespace parameter which is exposed to django. So what?

comment:6 Changed 9 years ago by Malcolm Tredinnick

Before the comments really get out of hand...

This is probably worth doing in some limited form. I'd guess using the existing DATABASE_OPTIONS setting is probably the most logical way, similar to how MySQL's engine setting is supported now. It's not worth adding it as an option to each model, since it's unlikely to vary between models (and for those cases where it does, the initial SQL support can be used). Two extra settings just for MySQL support is a little excessive, too.

I'd want to rejig DATABASE_OPTIONS a little bit first, though, so that, for example, we can differentiate between creation time options (engine, collation, etc) and general connection options. That will require some design work, but that's my thinking at the moment.

comment:7 Changed 9 years ago by Armin Ronacher

@mtredinnick: Sounds like a good idea.

comment:8 Changed 8 years ago by anonymous

Cc: lidaobing@… added

comment:9 Changed 6 years ago by Malcolm Tredinnick

Triage Stage: Design decision neededAccepted

comment:10 Changed 6 years ago by Armin Ronacher

Damn. That was not a very friendly written ticket. :)

comment:11 Changed 6 years ago by Armin Ronacher

In hindsight that should be implemented database backend independent and unicode does not appear to specify names for collations, so that should probably not be fixed in the way I originally recommended.

comment:12 Changed 6 years ago by Gabriel Hurley

Severity: Normal
Type: New feature

comment:13 Changed 5 years ago by jdchrist@…

Cc: jdchrist@… added
Easy pickings: unset
UI/UX: unset

comment:14 Changed 23 months ago by Collin Anderson

Cc: cmawebsite@… added
Keywords: mysql added

comment:15 Changed 23 months ago by Shai Berger

Cc: Shai Berger added

For the record:

All core backends (except Oracle, it seems) can set collation per column; collation names are not standard (and SQLite has only 3 built-in collations, none of them reasonable for any native language). Except (I think) for Oracle, they can also set collation for a specific query (the collation can be used in ordering comparisons or in ORDER BY). This means that any reasonable collation support in core would beget backend-specific usages, which I wouldn't like.

However, I think core should make it possible to add collation support via external libraries. I would like to see the API for such extension.

Of all core backends, the only one which can set the character encoding at a level lower than the whole database is MySQL. Since Django does not usually create databases (except for test databases), I think this feature should not be added (except, with the API mentioned above, in a MySQL-specific library outside core).

comment:16 Changed 14 months ago by Adam Chainz

Cc: me@… added
Note: See TracTickets for help on using tickets.
Back to Top