Restoring YAML fixtures with DateTimeField causes timezone warnings

[aigarius@…]

Loading a YAML fixture with a timezone aware DateTimeField in Django 1.4 will cause the following warning even if the field is defined with timezone information in the YAML file

django/db/models/fields/__init__.py:808: RuntimeWarning: DateTimeField received a naive datetime (2012-08-27 13:01:56.784734) while time zone support is active.

Excerpt from testdata.yaml

[...] , insert_date: !!timestamp '2012-08-27 13:01:56.784734+00:00', [...]

The test data is loaded by adding "fixtures = testdata" at the start of a test case class.

This bug is mentioned in Django test suite - django/trunk/tests/modeltests/timezones/tests.py:SerializationTests

Note: the trick of "MyTests = override_settings(USE_TZ=False)(MyTests)" (from #17940) often does not work as expected, because you can then get "ValueError: SQLite backend does not support timezone-aware datetimes when USE_TZ is False" during fixture loading

[Aymeric Augustin]

Indeed, this is ​a bug in PyYAML, as ​described in this comment.

There hasn't been any activity whatsoever on PyYAML's tracker, besides the creation of new tickets, since over one year. The project looks abandoned.

We aren't going to removing support for YAML fixtures now just because of this bug in PyYAML. We can document this as a known problem ​here and tell people to use JSON fixtures.

Other ideas?

[Russell Keith-Magee]

Historically, we haven't modified Django's docs to point out problems in downstream libraries -- the reasoning being that if the bug is fixed, there's never an action to correct Django's documentation and remove the warning. We've usually relied on the ticket tracker making it easy to find such problems.

[Aymeric Augustin]

Russell, what's your recommendation at this point? Just keep the ticket open until PyYAML gets fixed?

[Douglas Creager]

I've found a workaround that I can't find documented anywhere else, so I'm posting it here for the record.

The key is to force PyYAML to interpret the timestamp values as strings, and not let it perform the conversion to a datetime. PyYAML will then pass the bare string value on to Django, which will correctly parse a timezone-aware datetime.

For this to work you must make sure that all of the following are true:

1. Don't use the !!timestamp YAML tag
2. Enclose the timestamp values in quotes (which bypasses PyYAML's logic for automatically detecting a value's type based on what it looks like)
3. Include timezone information in the timestamp value

When we do all three of these steps with our fixture data, we don't see any naive datetime warnings. Note that this doesn't help any with #21578, since the data dumper produces YAML content that doesn't follow all three of these rules. We've had to resort to a manual post-processing step to massage the timestamp values. Not ideal, but it works.

[Michael Schwarz]

Replying to dcreager:

1. Don't use the !!timestamp YAML tag

Is there a way to prevent ./manage.py dumpdata --format yaml from automatically inserting the !!timestamp tags? We mainly run into this problem when using that command to generate fixtures for our unit tests.

[CHI Cheng]

PyYaml's (still open) pull request: ​https://github.com/yaml/pyyaml/pull/113