Opened 18 months ago

Last modified 18 months ago

#33143 new Cleanup/optimization

Block import-time queries

Reported by: Adam Johnson Owned by: nobody
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


It's possible to make ORM queries at import time. For example:

class Form(forms.Form):
    country = forms.ChoiceField(choices=[ for c in Country.objects.all()])

I see import time queries fairly frequently, and every time I have encountered them they were a mistake. They end up querying the database once when the web server starts and caching the data forever, which is never desired.

Python allows imports to happen at any time, so there's no way to detect when "import time" is over, and the app is really running. An inner import may cause a module to be first loaded during a web request.

I therefore propose we block queries until *after* the AppConfig.ready() phase. This would protect against most problems.

We could block until *before* AppConfig.ready(). Queries inside a ready() method kind-of imply the user knows that the query is one-off at startup. But we do encourage users to perform some imports within ready() methods such as for signal registration.

Change History (4)

comment:2 Changed 18 months ago by Simon Charette

I'd be in favor of blocking until *after* AppConfig.ready() phase as any queries performed at .ready() time has the potential to crash due to still unapplied migrations (missing table, schema mismatch) and can get users in a pickle where they can't even run migrate. This is something we've been warning against for a while and that I've had to point to a fair amount of time when reviewing Django project changes.

comment:3 Changed 18 months ago by Adam Johnson

Thanks for the developers discussion link Tim. I like the proposal of warning and then if there are no serious reports, deprecating from the next version. A RuntimeWarning sounds appropriate.

And I see you have some history with this Simon.

Aymeric was previously "+0" but he may have changed his mind. I will post again on the thread with the link to this ticket just to see if there are any dissenting opinions.

comment:4 Changed 18 months ago by Carlton Gibson

Triage Stage: UnreviewedAccepted

I think there's enough consensus to Accept.

Right or wrong, I suspect there are plenty of folks using ready() as a home for running some kind of query at startup. We should have some advice for them.

Note: See TracTickets for help on using tickets.
Back to Top