Opened 7 months ago
Last modified 7 months ago
#36484 closed New feature
Add optional error-on-integer-overflow setting for QuerySet filters — at Initial Version
| Reported by: | bubusuke | Owned by: | |
|---|---|---|---|
| Component: | Database layer (models, ORM) | Version: | dev |
| Severity: | Normal | Keywords: | |
| Cc: | bubusuke | Triage Stage: | Unreviewed |
| Has patch: | no | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description
Overview
In Django 5.0 the behaviour for arithmetic that overflows an integer column changed: a filter that would overflow now always returns an empty QuerySet instead of being transparently cast to a wider integer type (the historical behaviour on most back-ends). While this keeps runtime semantics simple, it makes migrations from Django 4.x harder to verify—tests may “pass” yet silently return fewer (or zero) rows. It is also difficult for developers on 5.x to notice when an overflow is happening because the condition is swallowed inside the ORM.
(see: https://docs.djangoproject.com/en/5.2/releases/5.0/#:~:text=Filtering%20querysets%20against%20overflowing%20integer%20values%20now%20always%20returns%20an%20empty%20queryset.%20As%20a%20consequence%2C%20you%20may%20need%20to%20use%20ExpressionWrapper()%20to%20explicitly%20wrap%20arithmetic%20against%20integer%20fields%20in%20such%20cases)
To improve debuggability and give teams a migration aid, I propose an opt-in setting that raises a specific exception the moment an overflow is detected.
Proposal
Add new parameter.
# settings.py ERROR_IF_INTEGER_OVERFLOW_IN_FILTER_QUERYSET = True # default: False
Implementation sketch
- Add new exception
class EmptyResultSetDueToIntegerOverflow(Exception): """Raised when an integer overflow is detected while building a WHERE clause. Subclasses plain Exception so it is *not* caught by existing `EmptyResultSet` handlers."""
- Raise it at the source
In django.db.models.lookups.IntegerFieldOverflow (or equivalent part of the lookup machinery), raise EmptyResultSetDueToIntegerOverflow instead of EmptyResultSet.
- Handle it centrally
In django.db.models.sql.where.WhereNode.as_sql() wrap the call to compiler.compile(child)
try:
...
except EmptyResultSetDueToIntegerOverflow:
if settings.ERROR_IF_INTEGER_OVERFLOW_IN_FILTER_QUERYSET:
raise
empty_needed -= 1 # current silent-empty logic
- Default behaviour unchanged
The setting defaults to False, so existing applications keep the current silent-empty semantics unless they explicitly enable strict mode.
Benefits
- Safe migrations – Projects upgrading from 4.x can enable the flag in CI to surface every place where arithmetic now truncates results.
- Debuggability – Developers on 5.x who run into an unexpected empty queryset get an immediate stack-trace pointing at the offending expression.
- Backwards-compatible – Opt-in toggle preserves the 5.0 behaviour by default; no existing code breaks unless the project explicitly asks for stricter checking.
- Path for future tightening – If the community agrees, the flag could default to True in a future major release after a full deprecation cycle.
I am happy to prepare a pull request with tests and documentation if this approach is acceptable. Thank you for considering this enhancement.