Opened 9 years ago

Last modified 10 months ago

#5745 new New feature

MySQL Collations/Charsets and Engines

Reported by: Armin Ronacher Owned by: nobody
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords: mysql
Cc: lidaobing@…, jdchrist@…, cmawebsite@…, shaib, me@… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


Django should really have a DATABASE_CHARSET / DATABASE_COLLATION setting in the as well as a setting for the mysql database engine. It's stunning that unittests can enforce a database collation but tables not. Also an argument in the Meta section of models that allows to define the collation, charset and engine would be an important thing to have.

I know that there are workarounds for those limitations but they have all disadvantages.

Change History (16)

comment:1 Changed 9 years ago by Simon G <dev@…>

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Design decision needed

comment:2 Changed 9 years ago by proxor

-0 to this. I think that Unicode should be used everythere by default.

comment:3 Changed 9 years ago by mitsuhiko

Unicode yes. But collations do not affect unicode but the way it deals with unicode data (german sorting, french sorting, greek sorting etc.). And other encodings in the database are required if you deal with old database data. And currently django creates latin1 tables which is the database default.

comment:4 Changed 9 years ago by proxor

Anyways this is a database-specific thing.

comment:5 Changed 9 years ago by mitsuhiko

So is the Oracle db_tablespace parameter which is exposed to django. So what?

comment:6 Changed 9 years ago by mtredinnick

Before the comments really get out of hand...

This is probably worth doing in some limited form. I'd guess using the existing DATABASE_OPTIONS setting is probably the most logical way, similar to how MySQL's engine setting is supported now. It's not worth adding it as an option to each model, since it's unlikely to vary between models (and for those cases where it does, the initial SQL support can be used). Two extra settings just for MySQL support is a little excessive, too.

I'd want to rejig DATABASE_OPTIONS a little bit first, though, so that, for example, we can differentiate between creation time options (engine, collation, etc) and general connection options. That will require some design work, but that's my thinking at the moment.

comment:7 Changed 9 years ago by mitsuhiko

@mtredinnick: Sounds like a good idea.

comment:8 Changed 8 years ago by anonymous

  • Cc lidaobing@… added

comment:9 Changed 6 years ago by mtredinnick

  • Triage Stage changed from Design decision needed to Accepted

comment:10 Changed 6 years ago by mitsuhiko

Damn. That was not a very friendly written ticket. :)

comment:11 Changed 6 years ago by mitsuhiko

In hindsight that should be implemented database backend independent and unicode does not appear to specify names for collations, so that should probably not be fixed in the way I originally recommended.

comment:12 Changed 5 years ago by gabrielhurley

  • Severity set to Normal
  • Type set to New feature

comment:13 Changed 5 years ago by jdchrist@…

  • Cc jdchrist@… added
  • Easy pickings unset
  • UI/UX unset

comment:14 Changed 19 months ago by collinanderson

  • Cc cmawebsite@… added
  • Keywords mysql added

comment:15 Changed 19 months ago by shaib

  • Cc shaib added

For the record:

All core backends (except Oracle, it seems) can set collation per column; collation names are not standard (and SQLite has only 3 built-in collations, none of them reasonable for any native language). Except (I think) for Oracle, they can also set collation for a specific query (the collation can be used in ordering comparisons or in ORDER BY). This means that any reasonable collation support in core would beget backend-specific usages, which I wouldn't like.

However, I think core should make it possible to add collation support via external libraries. I would like to see the API for such extension.

Of all core backends, the only one which can set the character encoding at a level lower than the whole database is MySQL. Since Django does not usually create databases (except for test databases), I think this feature should not be added (except, with the API mentioned above, in a MySQL-specific library outside core).

comment:16 Changed 10 months ago by adamchainz

  • Cc me@… added
Note: See TracTickets for help on using tickets.
Back to Top