Opened 3 years ago
Closed 13 months ago
#33143 closed Cleanup/optimization (fixed)
Block import-time queries
Reported by: | Adam Johnson | Owned by: | Florian Zimmermann |
---|---|---|---|
Component: | Database layer (models, ORM) | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Ready for checkin | |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
It's possible to make ORM queries at import time. For example:
class Form(forms.Form): country = forms.ChoiceField(choices=[c.name for c in Country.objects.all()])
I see import time queries fairly frequently, and every time I have encountered them they were a mistake. They end up querying the database once when the web server starts and caching the data forever, which is never desired.
Python allows imports to happen at any time, so there's no way to detect when "import time" is over, and the app is really running. An inner import may cause a module to be first loaded during a web request.
I therefore propose we block queries until *after* the AppConfig.ready()
phase. This would protect against most problems.
We could block until *before* AppConfig.ready()
. Queries inside a ready()
method kind-of imply the user knows that the query is one-off at startup. But we do encourage users to perform some imports within ready()
methods such as for signal registration.
Change History (15)
comment:1 by , 3 years ago
comment:2 by , 3 years ago
I'd be in favor of blocking until *after* AppConfig.ready()
phase as any queries performed at .ready()
time has the potential to crash due to still unapplied migrations (missing table, schema mismatch) and can get users in a pickle where they can't even run migrate
. This is something we've been warning against for a while and that I've had to point to a fair amount of time when reviewing Django project changes.
comment:3 by , 3 years ago
Thanks for the developers discussion link Tim. I like the proposal of warning and then if there are no serious reports, deprecating from the next version. A RuntimeWarning
sounds appropriate.
And I see you have some history with this Simon.
Aymeric was previously "+0" but he may have changed his mind. I will post again on the thread with the link to this ticket just to see if there are any dissenting opinions.
comment:4 by , 3 years ago
Triage Stage: | Unreviewed → Accepted |
---|
I think there's enough consensus to Accept.
Right or wrong, I suspect there are plenty of folks using ready()
as a home for running some kind of query at startup. We should have some advice for them.
comment:5 by , 15 months ago
Owner: | changed from | to
---|---|
Status: | new → assigned |
comment:7 by , 15 months ago
Patch needs improvement: | set |
---|
comment:8 by , 15 months ago
Patch needs improvement: | unset |
---|
comment:9 by , 14 months ago
Needs tests: | set |
---|---|
Patch needs improvement: | set |
comment:10 by , 14 months ago
Needs tests: | unset |
---|---|
Patch needs improvement: | unset |
comment:11 by , 13 months ago
Patch needs improvement: | set |
---|
Marking as need improvement per Adam's comment.
comment:12 by , 13 months ago
Patch needs improvement: | unset |
---|
comment:13 by , 13 months ago
Patch needs improvement: | set |
---|
comment:14 by , 13 months ago
Patch needs improvement: | unset |
---|---|
Triage Stage: | Accepted → Ready for checkin |
django-developers discussion