#19738 closed Bug (fixed)
"manage.py shell" on a fresh project raises RuntimeWarning about naive datetime, if IPython is installed
Reported by: | Carl Meyer | Owned by: | Aymeric Augustin |
---|---|---|---|
Component: | Database layer (models, ORM) | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Ready for checkin | |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
IPython uses SQLite internally to store shell session history. At the beginning and end of every shell session, it issues a SQLite query to get/save the history. For some reason this query includes a (timezone-naive) datetime.
Django's SQLite DB backend registers a global datetime adapter function which issues a warning in case it receives a naive datetime when timezone support is activated.
In combination, this means that on a fresh startproject
if you fire up manage.py shell
without making a single change to settings, if IPython is installed you'll get "RuntimeWarning: SQLite received a naive datetime (2013-02-04 17:14:13.070091) while time zone support is active." on shell startup and exit. That's an ugly experience for a newcomer to Django - it _looks_ to them as if Django is shipping a broken setup by default.
Probably this is Django's fault for registering a potentially-noisy process-global SQLite adapter, and we should consider whether to just remove that warning in the adapter. The model layer will already warn about naive datetimes, so this warning is only needed for raw SQL.
Change History (15)
comment:1 by , 12 years ago
Component: | Uncategorized → Database layer (models, ORM) |
---|---|
Type: | Uncategorized → Bug |
comment:2 by , 12 years ago
Triage Stage: | Unreviewed → Accepted |
---|
comment:3 by , 12 years ago
I made an attempt to temporarily hide the warning, shouldn't be much harmful. https://github.com/crodjer/django/commit/8b24546f2176d47a3443184728a71b2807d2a168
Quoting the commit message:
It is more of an ugly hack, required because of django injecting
global callback in a database api.
Ideally, instead of registering adapter for
datetime.datetime
(django.db.backends.sqlite3.base.py Line 76), django should do it
for a pseudo wrapper overdatetime.datetime
and use it everywhere,
so that other applications and libraries don't have to worry about
the behaviour being infected.
Also, while trying to debug this, I came across another similar issue (#19789) which involved a DeprecationWarning in IPython. It is probably irrelevant but still decided to link it here.
comment:4 by , 11 years ago
Has patch: | set |
---|---|
Version: | 1.4 → master |
comment:5 by , 11 years ago
I disagree with the approach in PR #1404 because it introduces a data corruption bug.
comment:6 by , 11 years ago
Hiding the warning appears to be the least awful solution at this point. crodjer's patch probably does the job.
comment:7 by , 11 years ago
Looking back at that PR, not sure what I was thinking other than maybe I had badly misread Carl's suggestion of removing the warning, as "removing the adapter" - other than that, I can't explain why I was taking that approach.
So to be clear, the main edge case we don't have an solution for, is raw sql issued by user's Django related code, that may use naive datetimes - as these will not generate any warnings as they would in the standard shell?
I think this is worth documenting/warning somehow - but is uncommon enough it shouldn't be too noisy verbose.
comment:8 by , 11 years ago
This warning is still present in:
$ python --version Python 2.7.5+ $ python -c "import django; print(django.get_version())" 1.5.4
I'm a python newcomer and I was looking a way to respect $PYTHONSTARTUP. Following #6682 I also installed IPython.
both ipython, and django came from ubuntu packages
$ python-django 1.5.4-1ubuntu1 High-level Python web development framework $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 13.10 Release: 13.10 Codename: saucy
this gives:
$ python manage.py shell /usr/lib/python2.7/dist-packages/django/db/backends/sqlite3/base.py:53: RuntimeWarning: SQLite received a naive datetime (2014-01-26 18:40:37.242040) while time zone support is active. RuntimeWarning) /usr/lib/python2.7/dist-packages/IPython/frontend/terminal/embed.py:239: DeprecationWarning: With-statements now directly support multiple context managers with nested(self.builtin_trap, self.display_trap): Python 2.7.5+ (default, Sep 19 2013, 13:49:51) Type "copyright", "credits" or "license" for more information. IPython 0.13.2 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. In [1]:
I found a workaround here: http://stackoverflow.com/questions/11708821/django-ipython-sqlite-complains-about-naive-datetime
But this fixes only the sqlite warning.
$ ipython profile create [ProfileCreate] Generating default config file: u'/home/sylvain/.config/ipython/profile_default/ipython_config.py'
and added those lines
import warnings import exceptions warnings.filterwarnings("ignore", category=exceptions.RuntimeWarning, module='django.db.backends.sqlite3.base', lineno=53)
rereading the tutorial
I found a better way (ipython removed):
export PYTHONSTARTUP=$PWD/_pythonrc export DJANGO_SETTINGS_MODULE=mysite.settings $ cat _pythonrc # Import the model classes we just wrote. from polls.models import Poll, Choice from django.utils import timezone $ python Python 2.7.5+ (default, Sep 19 2013, 13:49:51) [GCC 4.8.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> Poll.objects.all() [<Poll: What's up?>]
comment:9 by , 11 years ago
I took a look at applying crodjer's patch, but there are two issues I see with it:
1) It only solves half the problem. IPython sends a naive datetime to SQLite both at session start and session end. Crodjer's patch is careful to restore the warnings after session start, but there is still a warning raised at session end.
2) In the latest (1.0+) versions of IPython, the public embed API no longer gives us separate "init" and "start" methods, just a single start_ipython
method that does not return until the shell session is ended. This means that we can no longer disable the warning, initialize IPython, and then re-enable the warning before the user actually begins their shell interaction.
We can of course solve both of those problems by simply ignoring the warnings before starting IPython and never restoring them. But this means that someone using raw SQL via the IPython shell could save naive datetimes to their database without getting a warning, which re-opens a (small) window to the data-corruption bug that worried Aymeric above.
It seems that some users are fixing this for themselves by unconditionally disabling the warning in settings.py, which is much worse, because it means they could get silent data corruption anywhere in their code at any time, not only when using the IPython shell. So being purist about our response to this may end up putting at least some of our users in an even worse situation with regard to possible data corruption (though of course they are making an explicit choice to put themselves in that situation).
Another ugly alternative (of many ugly alternatives) is that we could fix this in at least recent IPython versions by actually monkeypatching IPython itself to send timezone-aware, rather than naive, datetimes at session start and end. This of course ties us to internal IPython implementation details with no backwards-compatibility guarantees.
In summary: I have no good suggestions for fixing this. I think it is Django who is being a badly-behaved library here by registering process-global adapters in the SQLite library (and/or SQLite is a bad library for exposing this adapter registration as a process-global API rather than a per-connection API), but it seems we are caught between a rock and a hard place.
comment:10 by , 11 years ago
Patch needs improvement: | set |
---|
comment:11 by , 10 years ago
Has patch: | unset |
---|---|
Owner: | changed from | to
Patch needs improvement: | unset |
Status: | new → assigned |
The landscape has changed quite a bit after e9103402c0fa873aea58a6a11dba510cd308cb84.
I'm working on simplifying conversions of datetimes read from the database. I may be able to fix this bug at the same time.
comment:13 by , 9 years ago
Triage Stage: | Accepted → Ready for checkin |
---|
I've also been experiencing this issue.