Opened 4 years ago

Closed 4 years ago

Last modified 5 months ago

#18867 closed Cleanup/optimization (invalid)

Restoring YAML fixtures with DateTimeField causes timezone warnings

Reported by: aigarius@… Owned by: aaugustin
Component: Database layer (models, ORM) Version: 1.4
Severity: Normal Keywords:
Cc:… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by russellm)

Loading a YAML fixture with a timezone aware DateTimeField in Django 1.4 will cause the following warning even if the field is defined with timezone information in the YAML file

django/db/models/fields/ RuntimeWarning: DateTimeField received a naive datetime (2012-08-27 13:01:56.784734) while time zone support is active.

Excerpt from testdata.yaml

[...] , insert_date: !!timestamp '2012-08-27 13:01:56.784734+00:00', [...]

The test data is loaded by adding "fixtures = testdata?" at the start of a test case class.

This bug is mentioned in Django test suite - django/trunk/tests/modeltests/timezones/

Note: the trick of "MyTests = override_settings(USE_TZ=False)(MyTests)" (from #17940) often does not work as expected, because you can then get "ValueError: SQLite backend does not support timezone-aware datetimes when USE_TZ is False" during fixture loading

Change History (7)

comment:1 Changed 4 years ago by aaugustin

  • Component changed from Database layer (models, ORM) to Documentation
  • Needs documentation unset
  • Needs tests unset
  • Owner changed from nobody to aaugustin
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted
  • Type changed from Bug to Cleanup/optimization

Indeed, this is a bug in PyYAML, as described in this comment.

There hasn't been any activity whatsoever on PyYAML's tracker, besides the creation of new tickets, since over one year. The project looks abandoned.

We aren't going to removing support for YAML fixtures now just because of this bug in PyYAML. We can document this as a known problem here and tell people to use JSON fixtures.

Other ideas?

comment:2 Changed 4 years ago by russellm

  • Description modified (diff)

Historically, we haven't modified Django's docs to point out problems in downstream libraries -- the reasoning being that if the bug is fixed, there's never an action to correct Django's documentation and remove the warning. We've usually relied on the ticket tracker making it easy to find such problems.

comment:3 Changed 4 years ago by aaugustin

  • Component changed from Documentation to Database layer (models, ORM)

Russell, what's your recommendation at this point? Just keep the ticket open until PyYAML gets fixed?

comment:4 Changed 4 years ago by aaugustin

  • Resolution set to invalid
  • Status changed from new to closed
Version 0, edited 4 years ago by aaugustin (next)

comment:5 follow-up: Changed 2 years ago by dcreager

I've found a workaround that I can't find documented anywhere else, so I'm posting it here for the record.

The key is to force PyYAML to interpret the timestamp values as strings, and not let it perform the conversion to a datetime. PyYAML will then pass the bare string value on to Django, which will correctly parse a timezone-aware datetime.

For this to work you must make sure that all of the following are true:

  1. Don't use the !!timestamp YAML tag
  2. Enclose the timestamp values in quotes (which bypasses PyYAML's logic for automatically detecting a value's type based on what it looks like)
  3. Include timezone information in the timestamp value

When we do all three of these steps with our fixture data, we don't see any naive datetime warnings. Note that this doesn't help any with #21578, since the data dumper produces YAML content that doesn't follow all three of these rules. We've had to resort to a manual post-processing step to massage the timestamp values. Not ideal, but it works.

comment:6 in reply to: ↑ 5 Changed 5 months ago by Feuermurmel

Replying to dcreager:

  1. Don't use the !!timestamp YAML tag

Is there a way to prevent ./ dumpdata --format yaml from automatically inserting the !!timestamp tags? We mainly run into this problem when using that command to generate fixtures for our unit tests.

comment:7 Changed 5 months ago by Feuermurmel

  • Cc… added
Note: See TracTickets for help on using tickets.
Back to Top