#36475 closed Cleanup/optimization (needsinfo)
Optimize F-expression slice to MySQL column-prefix index
| Reported by: | JaeHyuckSa | Owned by: | |
|---|---|---|---|
| Component: | Database layer (models, ORM) | Version: | 5.1 |
| Severity: | Normal | Keywords: | |
| Cc: | Adam Johnson | Triage Stage: | Unreviewed |
| Has patch: | no | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description (last modified by )
Since Django 5.1 you can do this in your model
class MyModel(models.Model):
field = models.CharField(max_length=200)
class Meta:
indexes = [
Index(F('field')[:10], name='prefix_idx'),
]
However, on all backends this still creates:
CREATE INDEX prefix_idx ON myapp_mymodel (SUBSTRING(field, 1, 10));
MySQL supports a more efficient “column prefix” syntax:
CREATE INDEX prefix_idx ON myapp_mymodel (field(10));
This syntax is more efficient and directly supports LIKE and startswith queries on long VARCHAR or TEXT columns.
MySQL docs: https://dev.mysql.com/doc/refman/9.3/en/create-index.html#create-index-column-prefixes
This has come up before — see Ticket #35777 — and was also discussed in django-mysql # https://github.com/adamchainz/django-mysql/pull/1151#issuecomment-2995608422 where Adam Johnson suggested that Django could handle this directly instead of needing custom index subclasses.
Change History (4)
comment:1 by , 5 months ago
| Description: | modified (diff) |
|---|
comment:2 by , 5 months ago
| Cc: | added |
|---|
comment:3 by , 5 months ago
| Resolution: | → needsinfo |
|---|---|
| Status: | new → closed |
comment:4 by , 5 months ago
The idea is that MySQL could have a custom _create_index_sql() that replaces relevant Substr instances with another expression that then outputs the col_name(length) syntax.
Perhaps this is too complicated, and it's better to make a separate Prefix expression class, though, in Django-MySQL.
Unless I have misunderstood, which is very possible, I don't see how this can be implemented without a new index class like
PrefixIndexwhich was suggested within #35777 and I see is what you have written for django-mysqlSlicing an
Fexpression of aCharFieldresolves to aSubstrexpression. The MySQL syntaxcol_name(length)is (from what I can tell) is index specific and so not we won't be able to override the sql for MySQL withinSubstr. In the PR discussion you mentioned overriding_create_index_sqlbut I can't envision what you are thinking. If you have a draft PR to propose this I might be able to see what you mean?Note that #16460 is almost a duplicate