Opened 3 months ago
Closed 2 months ago
#35683 closed Bug (wontfix)
django.utils.timezone.make_naive can underflow for timezones close to datetime.min
Reported by: | Liam DeVoe | Owned by: | Shubham Singh Sugara |
---|---|---|---|
Component: | Database layer (models, ORM) | Version: | 5.1 |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Accepted | |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description (last modified by )
I'm a contributor over at Hypothesis. We found that saving (or retrieving?) models with aware DateTimeField values close to datetime.min can error if:
- timezone support is enabled (USE_TZ=True)
- you are using the sqlite database (presumably also any other db backend without tz aware support, if they exist?)
Explicitly:
from datetime import datetime, timezone from zoneinfo import ZoneInfo from django.utils.timezone import make_naive dt = datetime.min.replace(tzinfo=ZoneInfo(key='Africa/Addis_Ababa')) make_naive(dt, timezone.utc) # OverflowError
where make_naive is called from here. I would guess the overflow case for datetime.max is symmetric and similar as well.
I hope you'll forgive the lack of a django reproducer - I live in testing land and don't have a local django environment handy :). Hopefully the cause is clear and reproducible with vanilla django models.
Change History (8)
comment:1 by , 3 months ago
Description: | modified (diff) |
---|
comment:2 by , 3 months ago
Component: | Uncategorized → Database layer (models, ORM) |
---|---|
Triage Stage: | Unreviewed → Accepted |
comment:3 by , 3 months ago
Owner: | set to |
---|---|
Status: | new → assigned |
will try to work on it but will have to check first
Bug is causing issue on other db as well if so look for fix
follow-up: 5 comment:4 by , 3 months ago
What behavior do you expect? If the timezone conversion causes the datetime to be too small or too big, what can be done?
comment:5 by , 3 months ago
Replying to Tim Graham:
What behavior do you expect? If the timezone conversion causes the datetime to be too small or too big, what can be done?
For now i can think of this solution
Max case should be also included if datetime.min is giving error than there is a possibility datetime.max also giving the error due to corner cases
We can Update make_naive Function:
Modify the make_naive function in Django to handle boundary cases(max and min). For instance, if an OverflowError is detected, the function could return the closest safe datetime value or handle the error gracefully in another way.
for now i think this can work across all db as changing other parts of modules will be db specific
but if any other suggestion open to discussion
comment:6 by , 3 months ago
The use case for storing datetime.min
and max
is unclear, but it seems doubtful that we should silently change the value the user provided. I would close this ticket as "needs info" or "wontfix" unless the reporter can provide more explanation.
comment:7 by , 3 months ago
I half expected this ticket to be wontfix when I opened it :). Here's one sane alternative I can think of: if you're converting to naive solely for serializing to the db, and when you deserialize from the db you make it tz aware again, then the only time it's out of bounds for the serialization/storage portion. But you control the serialization format, so you could extend the limits by 24 hours in each direction to accommodate those overflows, relying on the invariant that they will be made immediately tz aware and thus in bounds before reaching the user.
I'm not familiar with django internals, so this may not be acceptable or the assumed invariants may not be correct - that's up to you guys. I agree that silently capping is not a good idea. Possibly this should just be wontifx'd but documented; it does seem surprising to a user that I cannot store an in-bounds tz aware datetime, because there is no reason to expect that django will convert it to a naive datetime.
comment:8 by , 2 months ago
Resolution: | → wontfix |
---|---|
Status: | assigned → closed |
Thank you both for the discussion!
Given I believe this was found and validated via testing scenarios (rather than "in real life"), and much of this code has been this way for years, going to update to wontfix.
If anyone finds that this causes them an issue with their application, they're welcome to add this discussion.
Replicated, thank you!
Should be an issue in MySQL and Oracle as well - refs #23820
Rough testcase:
tests/backends/sqlite/test_operations.py