#36484 closed New feature (wontfix)
Add optional error-on-integer-overflow setting for QuerySet filters
Reported by: | bubusuke | Owned by: | |
---|---|---|---|
Component: | Database layer (models, ORM) | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | bubusuke | Triage Stage: | Unreviewed |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description (last modified by )
Overview
In Django 5.0 the behaviour for arithmetic that overflows an integer column changed: a filter that would overflow now always returns an empty QuerySet instead of being transparently cast to a wider integer type (the historical behaviour on most back-ends). While this keeps runtime semantics simple, it makes migrations from Django 4.x harder to verify—tests may “pass” yet silently return fewer (or zero) rows. It is also difficult for developers on 5.x to notice when an overflow is happening because the condition is swallowed inside the ORM.
(see: https://docs.djangoproject.com/en/5.2/releases/5.0/#:~:text=Filtering%20querysets%20against%20overflowing%20integer%20values%20now%20always%20returns%20an%20empty%20queryset.%20As%20a%20consequence%2C%20you%20may%20need%20to%20use%20ExpressionWrapper()%20to%20explicitly%20wrap%20arithmetic%20against%20integer%20fields%20in%20such%20cases)
To improve debuggability and give teams a migration aid, I propose an opt-in setting that raises a specific exception the moment an overflow is detected.
Proposal
Add new parameter.
# settings.py ERROR_IF_INTEGER_OVERFLOW_IN_FILTER_QUERYSET = True # default: False
Implementation sketch
- Add new exception
class EmptyResultSetDueToIntegerOverflow(Exception): """Raised when an integer overflow is detected while building a WHERE clause. Subclasses plain Exception so it is *not* caught by existing `EmptyResultSet` handlers."""
- Raise it at the source
In django.db.models.lookups.IntegerFieldOverflow (or equivalent part of the lookup machinery), raise EmptyResultSetDueToIntegerOverflow instead of EmptyResultSet.
- Handle it centrally
In django.db.models.sql.where.WhereNode.as_sql() wrap the call to compiler.compile(child)
try: ... except EmptyResultSetDueToIntegerOverflow: if settings.ERROR_IF_INTEGER_OVERFLOW_IN_FILTER_QUERYSET: raise empty_needed -= 1 # current silent-empty logic
- Default behaviour unchanged
The setting defaults to False, so existing applications keep the current silent-empty semantics unless they explicitly enable strict mode.
Benefits
- Safe migrations – Projects upgrading from 4.x can enable the flag in CI to surface every place where arithmetic now truncates results.
- Debuggability – Developers on 5.x who run into an unexpected empty queryset get an immediate stack-trace pointing at the offending expression.
- Backwards-compatible – Opt-in toggle preserves the 5.0 behaviour by default; no existing code breaks unless the project explicitly asks for stricter checking.
- Path for future tightening – If the community agrees, the flag could default to True in a future major release after a full deprecation cycle.
I am happy to prepare a pull request with tests and documentation if this approach is acceptable. Thank you for considering this enhancement.
Change History (3)
comment:1 by , 2 months ago
Description: | modified (diff) |
---|
comment:2 by , 2 months ago
Resolution: | → wontfix |
---|---|
Status: | new → closed |
Version: | 5.0 → dev |
comment:3 by , 2 months ago
Hi Natalia,
Thank you for the clear guidance.
I’m still getting used to the contribution workflow and didn’t realize the proper submission path, but I understand now. I’ll draft the proposal in the GitHub new-feature ideas project.
For the record, I didn’t use any AI tools when preparing the idea.
Thanks again.
Hello bubusuke! Thank you for the detailed and thoughtful proposal.
Considering that this is a new feature request, and per our current documented process for proposing new features, it should be submitted to the new feature ideas GitHub project, rather than this ticket tracker. That space is used for community discussion and Steering Council review prior to any decision and ultimately implementation work.
A small note: adding new settings is generally considered a high bar in Django and often controversial, so any proposal involving one will need strong justification and community support. Also, if you used any AI tools to assist in drafting this proposal, now and in the future we'd appreciate a brief note to that effect, just to help us assess how best to review and engage with the content.
Following standard practice, I'm closing this ticket as
wontfix
for now. If the idea gains support through the feature ideas process, it can be revisited.