Refresh fields that are expressions after Model.save()

[holvianssi]

The use case is automatically fetching the value for expressions when saving to DB. For example:

> user = User.objects.create(username=Lower('Anssi'))
> user.username == 'anssi'
True

While the above feature can be simulated somewhat easily by calling refresh_from_db() after save, an in-built implementation has the ability to use RETURNING as an optimization. In addition, it seems that refreshing objects on save would be a nice default, though this might be a bit backwards incompatible in some cases.

[Tim Graham]

Does this also fix #23386? In that case, we might close this as a duplicate.

[Simon Charette]

​PR

[Simon Charette]

This might also be related to #21454 or at least to the implementation proposed in it's ​PR as we should really be using RETURNING on backends that support it instead of performing a second SELECT to fetch the database generated fields.

[holvianssi]

I'm hesitant to go with RETURNING implementation for the first patch. The select approach is really simple, and it will be needed in any case for some backends. The RETURNING approach on the other hand will be complex, and after all it's just an optimisation.

This should almost solve #21454 with a Default expression. This would need to be assigned manually to fields pre-save. Then full solution to #21454 would be adding a bit of API to do the pre-save assignments automatically.

And yes, I believe this solves #23386.

[Tim Graham]

Comments for improvement on the PR.

[Tim Graham]

I update the PR per my comments.

[Tim Graham]

Simon still has concerns described on the pull request.

[Simon Charette]

Updated ​patch that makes use of RETURNING on backends that support it and clear the attributes otherwise (allowing refresh_from_db to kick in on attribute access). Still needs a few tweaks.

[Mariusz Felisiak <felisiak.mariusz@…>]

In dc4ee99:

Refs #27222 -- Implemented BaseDatabaseOperations.return_insert_columns()/fetch_returned_insert_rows().

[Mariusz Felisiak <felisiak.mariusz@…>]

In 292b9e6f:

Refs #27222 -- Adapted RETURNING handling to be usable for UPDATE queries.

Renamed existing methods and abstractions used for INSERT … RETURNING
to be generic enough to be used in the context of UPDATEs as well.

This also consolidates SQL compliant implementations on
BaseDatabaseOperations.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 55a0073b:

Refs #27222 -- Refreshed GeneratedFields values on save() initiated update.

This required implementing UPDATE RETURNING machinery that heavily
borrows from the INSERT one.

[Mariusz Felisiak <felisiak.mariusz@…>]

In 9468043:

Fixed #27222 -- Refreshed model field values assigned expressions on save().

Removed the can_return_columns_from_insert skip gates on existing
field_defaults tests to confirm the expected number of queries are
performed and that returning field overrides are respected.

[Jacob Walls <jacobtylerwalls@…>]

In e059bbe:

Refs #27222 -- Deduplicated db_returning fields in Model.save().

Follow-up to 94680437a45a71c70ca8bd2e68b72aa1e2eff337.

[Jacob Walls <jacobtylerwalls@…>]

In 4fcc2883:

Refs #27222 -- Restored Model.save()'s refreshing of db_returning fields even if a value is set.

The logic could likely be adjusted to assign the pre_save value in most cases
to avoid the database transit but it could break in subtle ways so it's not
worth the complexity it would require.

Regression in 94680437a45a71c70ca8bd2e68b72aa1e2eff337.

Co-authored-by: Tim Graham <timograham@…>