Opened 17 months ago

Closed 3 weeks ago

Last modified 3 weeks ago

#29444 closed Cleanup/optimization (fixed)

Allow fields to be part of the RETURNING clause during INSERT

Reported by: Johannes Hoppe Owned by: Johannes Hoppe
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords: django, db, returning, default, model, field
Cc: info@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Dependency for #27452

The SQL RETURNING statement is currently used for inserts of new objects. Mainly it is responsible to return the primary key in a single query.

That's great but currently in the postgres database backend this value is hard coded to only the primary key. I would like to propose to make this configurable.

There are plenty of uses cases for such a feature ultimately even enabling database defaults, but I would suggest to keep it very simple for now.

I would suggest to simply allow model fields to specify if they should be included in the returning statement or not. The default being not (except PK) to maintain the current behavior.

The implementation as well as the related code lines have been already discussed here:
https://github.com/django/django/pull/7525#issuecomment-354269077

I highly suggest to keep this a Postgres ONLY feature for now. That way we can see what sticks and consider extending this functionality into django.db in the future after we have seen how it's being used and gathered feedback.

Attachments (1)

refs-29444-oracle-part.diff (8.2 KB) - added by felixxm 5 weeks ago.

Download all attachments as: .zip

Change History (29)

comment:1 Changed 17 months ago by Johannes Hoppe

Version: 2.0master

comment:2 Changed 17 months ago by Johannes Hoppe

Owner: set to Johannes Hoppe
Status: newassigned

comment:3 Changed 17 months ago by Johannes Hoppe

Has patch: set
Last edited 17 months ago by Tim Graham (previous) (diff)

comment:4 Changed 17 months ago by Johannes Hoppe

Patch needs improvement: set
Triage Stage: UnreviewedAccepted

comment:5 Changed 14 months ago by Johannes Hoppe

Patch needs improvement: unset

comment:6 Changed 14 months ago by Tim Graham

Patch needs improvement: set

comment:7 Changed 14 months ago by Johannes Hoppe

Patch needs improvement: unset
Type: New featureCleanup/optimization

comment:8 Changed 13 months ago by Tim Graham

Component: contrib.postgresDatabase layer (models, ORM)
Patch needs improvement: set
Summary: Allow modification of RETURNING values in django.contrib.postgresAllow fields to be part of the RETURNING clause during INSERT

A number of Oracle test failures remain.

comment:9 Changed 10 months ago by Johannes Hoppe

Patch needs improvement: unset

I fixed a couple of things. Nothing fails for me now locally that doesn't fail on master. Can you trigger it again?

comment:10 Changed 9 months ago by Tim Graham <timograham@…>

In b131f9c7:

Refs #29444 -- Renamed DatabaseFeatures.can_return_id* to be generic for other columns.

comment:11 Changed 9 months ago by Tim Graham

Patch needs improvement: set

You should be able to trigger the Oracle build yourself as described on wiki:Jenkins. I ran the tests locally and put the failures in a PR comment.

comment:12 Changed 8 months ago by Johannes Hoppe

Patch needs improvement: unset

comment:13 Changed 4 months ago by Carlton Gibson

Needs tests: set

comment:14 Changed 4 months ago by Johannes Hoppe

Needs tests: unset

comment:15 Changed 3 months ago by felixxm

Patch needs improvement: set

comment:16 Changed 3 months ago by Johannes Hoppe

Patch needs improvement: unset

comment:17 Changed 3 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In bc91f27a:

Refs #29444 -- Added support for fetching a returned non-integer insert values on Oracle.

This is currently not actively used, since the ORM will ask the
SQL compiler to only return auto fields.

comment:18 Changed 3 months ago by felixxm

Patch needs improvement: set

comment:19 Changed 3 months ago by Johannes Hoppe

Patch needs improvement: unset

comment:20 Changed 6 weeks ago by Mariusz Felisiak <felisiak.mariusz@…>

In 003bb34b:

Refs #29444 -- Made db.backends.oracle.utils.InsertVar use str as default.

comment:21 Changed 6 weeks ago by felixxm

Triage Stage: AcceptedReady for checkin

comment:22 Changed 5 weeks ago by Mariusz Felisiak <felisiak.mariusz@…>

In 736e7d44:

Refs #29444 -- Fixed DateField constructor in db.backends.oracle.utils.InsertVar.

comment:23 Changed 5 weeks ago by Mariusz Felisiak <felisiak.mariusz@…>

In 7254f113:

Refs #29444 -- Allowed returning multiple fields from INSERT statements on PostgreSQL.

Thanks Florian Apolloner, Tim Graham, Simon Charette, Nick Pope, and
Mariusz Felisiak for reviews.

Changed 5 weeks ago by felixxm

Attachment: refs-29444-oracle-part.diff added

comment:24 Changed 5 weeks ago by felixxm

Has patch: unset
Triage Stage: Ready for checkinAccepted

This ticket is doable on Oracle with cx_Oracle 6.0+ that's why I'm leaving it open. First attempt of fixing this on Oracle was a part of the original PR (see a diff).

Last edited 4 weeks ago by felixxm (previous) (diff)

comment:25 Changed 4 weeks ago by felixxm

Has patch: set
Patch needs improvement: set

comment:26 Changed 4 weeks ago by Johannes Hoppe

Patch needs improvement: unset

comment:27 Changed 3 weeks ago by Mariusz Felisiak <felisiak.mariusz@…>

Resolution: fixed
Status: assignedclosed

In b31e6387:

Fixed #29444 -- Allowed returning multiple fields from INSERT statements on Oracle.

comment:28 Changed 3 weeks ago by Mariusz Felisiak <felisiak.mariusz@…>

In 0110436:

Refs #29444 -- Removed redundant DatabaseFeatures.can_return_multiple_columns_from_insert.

Unnecessary since b31e63879eb5d9717e9f890401f7222e4f00c910.

Note: See TracTickets for help on using tickets.
Back to Top