Opened 4 years ago

Closed 4 years ago

Last modified 8 months ago

#18867 closed Cleanup/optimization (invalid)

Restoring YAML fixtures with DateTimeField causes timezone warnings

Reported by: aigarius@… Owned by: Aymeric Augustin
Component: Database layer (models, ORM) Version: 1.4
Severity: Normal Keywords:
Cc: michi.schwarz@… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Russell Keith-Magee)

Loading a YAML fixture with a timezone aware DateTimeField in Django 1.4 will cause the following warning even if the field is defined with timezone information in the YAML file

django/db/models/fields/__init__.py:808: RuntimeWarning: DateTimeField received a naive datetime (2012-08-27 13:01:56.784734) while time zone support is active.

Excerpt from testdata.yaml

[...] , insert_date: !!timestamp '2012-08-27 13:01:56.784734+00:00', [...]

The test data is loaded by adding "fixtures = testdata?" at the start of a test case class.

This bug is mentioned in Django test suite - django/trunk/tests/modeltests/timezones/tests.py:SerializationTests

Note: the trick of "MyTests = override_settings(USE_TZ=False)(MyTests)" (from #17940) often does not work as expected, because you can then get "ValueError: SQLite backend does not support timezone-aware datetimes when USE_TZ is False" during fixture loading

Change History (7)

comment:1 Changed 4 years ago by Aymeric Augustin

Component: Database layer (models, ORM)Documentation
Owner: changed from nobody to Aymeric Augustin
Triage Stage: UnreviewedAccepted
Type: BugCleanup/optimization

Indeed, this is a bug in PyYAML, as described in this comment.

There hasn't been any activity whatsoever on PyYAML's tracker, besides the creation of new tickets, since over one year. The project looks abandoned.

We aren't going to removing support for YAML fixtures now just because of this bug in PyYAML. We can document this as a known problem here and tell people to use JSON fixtures.

Other ideas?

comment:2 Changed 4 years ago by Russell Keith-Magee

Description: modified (diff)

Historically, we haven't modified Django's docs to point out problems in downstream libraries -- the reasoning being that if the bug is fixed, there's never an action to correct Django's documentation and remove the warning. We've usually relied on the ticket tracker making it easy to find such problems.

comment:3 Changed 4 years ago by Aymeric Augustin

Component: DocumentationDatabase layer (models, ORM)

Russell, what's your recommendation at this point? Just keep the ticket open until PyYAML gets fixed?

comment:4 Changed 4 years ago by Aymeric Augustin

Resolution: invalid
Status: newclosed
Version 0, edited 4 years ago by Aymeric Augustin (next)

comment:5 Changed 2 years ago by Douglas Creager

I've found a workaround that I can't find documented anywhere else, so I'm posting it here for the record.

The key is to force PyYAML to interpret the timestamp values as strings, and not let it perform the conversion to a datetime. PyYAML will then pass the bare string value on to Django, which will correctly parse a timezone-aware datetime.

For this to work you must make sure that all of the following are true:

  1. Don't use the !!timestamp YAML tag
  2. Enclose the timestamp values in quotes (which bypasses PyYAML's logic for automatically detecting a value's type based on what it looks like)
  3. Include timezone information in the timestamp value

When we do all three of these steps with our fixture data, we don't see any naive datetime warnings. Note that this doesn't help any with #21578, since the data dumper produces YAML content that doesn't follow all three of these rules. We've had to resort to a manual post-processing step to massage the timestamp values. Not ideal, but it works.

comment:6 in reply to:  5 Changed 8 months ago by Michael Schwarz

Replying to dcreager:

  1. Don't use the !!timestamp YAML tag

Is there a way to prevent ./manage.py dumpdata --format yaml from automatically inserting the !!timestamp tags? We mainly run into this problem when using that command to generate fixtures for our unit tests.

comment:7 Changed 8 months ago by Michael Schwarz

Cc: michi.schwarz@… added
Note: See TracTickets for help on using tickets.
Back to Top