Improve docs for auto_now & auto_now_add and timezone handling

[djbug]

auto_now & auto_now_add ignore TIME_ZONE / USE_TZ settings for DateField

​https://github.com/django/django/blob/1.7/django/db/models/fields/__init__.py#L1146

    def pre_save(self, model_instance, add):
        if self.auto_now or (self.auto_now_add and add):
            value = datetime.date.today()
            setattr(model_instance, self.attname, value)
            return value
        else:
            return super(DateField, self).pre_save(model_instance, add)

Instead of datetime.date.today(), this should use timezone.now().date().

[Simon Charette]

Instead of datetime.date.today(), this should use timezone.now().date()

I think what you want is timezone.localtime(timezone.now()).date() instead since timezone.now() always returns a datetime in UTC.

IMO timezone.localtime(timezone.now()).date() if settings.USE_TZ else datetime.date.today() would still be a better default then the current one.

But I have to admit I didn't even know auto_now and auto_now_add worked with DateField so I'm not sure if we should go forward with this.

[djbug]

Replying to charettes:

IMO timezone.localtime(timezone.now()).date() if settings.USE_TZ else datetime.date.today() would still be a better default then the current one.

The ​docs say,

when support for time zones is enabled, Django stores date and time information in UTC in the database

Hence, if USE_TZ is True, shouldn't we store the date in UTC ?

[Simon Charette]

Replying to djbug:

Hence, if USE_TZ is True, shouldn't we store the date in UTC ?

The thing is the timezone is actually part on the data type used to store DateTimeField but there's no such thing for the date one.

I guess this is going to end up as a documentation clarification since I doubt we can change it either way without breaking backward compatibility.

[Aymeric Augustin]

In Django 1.4, with my original design, the model layer operated in the default time zone.

In Django 1.6, with in-database support for time zone-aware aggregation, we've started moving towards making the model layer operate in the current time zone.

The solution proposed in the original report is wrong. It will always return the date for the current datetime in UTC. The solution proposed in comment 2 makes more sense.

That said, until now DateField was safe from timezone-related issues. With that change, saving an object with auto_now or auto_now_add during the autumn DST change will raise an exception, because the datetime is ambiguous, even though the date isn't ambiguous.

My inclination would be to document the current behavior, in order to avoid a backwards-incompatible change, and recommend using ​Field.default instead.

(On a related note, we've been talking about deprecating auto_now and auto_now_add for years... so users could write default=datetime.date.today or something more elaborate).

[Tim Graham]

See #22995 for the ticket on that deprecation. It got some pushback so the effort stalled.

[Carl Meyer]

Replying to aaugustin:

That said, until now DateField was safe from timezone-related issues. With that change, saving an object with auto_now or auto_now_add during the autumn DST change will raise an exception, because the datetime is ambiguous, even though the date isn't ambiguous.

Of course, because we know that the date is not in fact ambiguous during a transition regardless of whether you force DST or no-DST, I think this would be a case where it would be safe to simply pass is_dst=False to the conversion and avoid the risk of exception.

My inclination would be to document the current behavior, in order to avoid a backwards-incompatible change, and recommend using ​Field.default instead.

But I am OK with this solution. If you actually care about timezones, I think you are probably better off in control of an explicit default.

[djbug]

Excuse my ignorance here. The ​docs for timezone say

When support for time zones is enabled, Django stores date and time information in UTC in the database [...] it’s still good practice to store data in UTC in your database

This makes me think that if USE_TZ is True, the date should be grabbed from the UTC datetime.

[Carl Meyer]

Replying to djbug:

This makes me think that if USE_TZ is True, the date should be grabbed from the UTC datetime.

That sentence in the docs should probably be updated for clarity to be specifically discussing "datetimes", not "dates and times".

It's good practice to store -datetimes- in the database as UTC, simply for consistency. That way you always know unambiguously what you're getting from the database, and a UTC datetime can always be converted to whatever timezone you need to display it in.

Dates and times are fundamentally different from datetimes -- they don't have a timezone, and they can't be converted into another timezone for display. (Well, a time can sort of be converted to a different timezone by just adding/subtracting an offset, if you're willing to be completely naive about DST, but that's generally a bad idea.) So the best-practice regarding datetimes does not apply to dates or times - it's not necessarily correct to just "always store in UTC" because they can't be converted from UTC to anything else, so what you should store in the database is "whatever is correct for your application." For the specific case of a DateField with auto_now(_add), what is correct really depends on the purpose of the field. If the server wants to consistently privately track the created-date for objects in a way that is comparable to each other, it may make sense to use "creation date in UTC", or in the server timezone (though really for this case I don't know why you'd use a DateField rather than a full timestamp). If it's a field meant for user display, then it probably makes more sense to use "today's date in the user's local timezone," otherwise it will appear to the user to give the wrong date sometimes.

My conclusions from all this:

• Unless you really know that you need a DateField, you should probably consider just using a DateTimeField instead, and convert to date for display only. Storing a DateField in the database is choosing to store a lossy and ambiguous format over a precise and unambiguous one. (Even if you need to compare dates in the database as dates, with modern Django lookups should make that possible if you store datetimes.)

• DateField(auto_now[_add]) is basically ambiguous when it comes to timezones - it would be better to be explicit with a default value that does what you expect regarding timezones. This might be a reason to consider deprecating auto_now(_add) for DateField only.

• Short of that, although it may be easier to imagine cases where you'd want "date in user's timezone" rather than "date in a way that's comparable across all objects", we don't really know which the user wants, and so we should default to not changing the current behavior. For instance, I could imagine someone using DateField(auto_now_add=True) for easily grouping objects by the day they were created on. This use case depends on the same timezone being used to determine the created-date for all objects created, so the proposal to switch to date-in-localtime would break it.

[Markus Holtermann]

Fixed in 8119876d4a533fbc2ba4d1c30eaddbcc28119488

Improved docs for timezone handling for auto_now and auto_now_add

[Markus Holtermann <info@…>]

In c4e8f21a:

[1.8.x] Fixed #24281 -- Improved docs for timezone handling for auto_now and auto_now_add

Thanks djbug for the report and Aymeric Augustin and Carl Meyer for the
review.

Backport of 8119876d4a533fbc2ba4d1c30eaddbcc28119488 from master

[Markus Holtermann <info@…>]

In 19ddf59b:

[1.7.x] Fixed #24281 -- Improved docs for timezone handling for auto_now and auto_now_add

Thanks djbug for the report and Aymeric Augustin and Carl Meyer for the
review.

Backport of 8119876d4a533fbc2ba4d1c30eaddbcc28119488 from master