Opened 19 months ago

Closed 3 months ago

Last modified 3 months ago

#29444 closed Cleanup/optimization (fixed)

Allow fields to be part of the RETURNING clause during INSERT

Reported by: Johannes Hoppe Owned by: Johannes Hoppe
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords: django, db, returning, default, model, field
Cc: info@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Dependency for #27452

The SQL RETURNING statement is currently used for inserts of new objects. Mainly it is responsible to return the primary key in a single query.

That's great but currently in the postgres database backend this value is hard coded to only the primary key. I would like to propose to make this configurable.

There are plenty of uses cases for such a feature ultimately even enabling database defaults, but I would suggest to keep it very simple for now.

I would suggest to simply allow model fields to specify if they should be included in the returning statement or not. The default being not (except PK) to maintain the current behavior.

The implementation as well as the related code lines have been already discussed here:
https://github.com/django/django/pull/7525#issuecomment-354269077

I highly suggest to keep this a Postgres ONLY feature for now. That way we can see what sticks and consider extending this functionality into django.db in the future after we have seen how it's being used and gathered feedback.

Attachments (1)

refs-29444-oracle-part.diff (8.2 KB) - added by felixxm 3 months ago.

Download all attachments as: .zip

Change History (29)

comment:1 Changed 19 months ago by Johannes Hoppe

Version: 2.0master

comment:2 Changed 19 months ago by Johannes Hoppe

Owner: set to Johannes Hoppe
Status: newassigned

comment:3 Changed 19 months ago by Johannes Hoppe

Has patch: set
Last edited 19 months ago by Tim Graham (previous) (diff)

comment:4 Changed 19 months ago by Johannes Hoppe

Patch needs improvement: set
Triage Stage: UnreviewedAccepted

comment:5 Changed 16 months ago by Johannes Hoppe

Patch needs improvement: unset

comment:6 Changed 16 months ago by Tim Graham

Patch needs improvement: set

comment:7 Changed 16 months ago by Johannes Hoppe

Patch needs improvement: unset
Type: New featureCleanup/optimization

comment:8 Changed 15 months ago by Tim Graham

Component: contrib.postgresDatabase layer (models, ORM)
Patch needs improvement: set
Summary: Allow modification of RETURNING values in django.contrib.postgresAllow fields to be part of the RETURNING clause during INSERT

A number of Oracle test failures remain.

comment:9 Changed 12 months ago by Johannes Hoppe

Patch needs improvement: unset

I fixed a couple of things. Nothing fails for me now locally that doesn't fail on master. Can you trigger it again?

comment:10 Changed 11 months ago by Tim Graham <timograham@…>

In b131f9c7:

Refs #29444 -- Renamed DatabaseFeatures.can_return_id* to be generic for other columns.

comment:11 Changed 11 months ago by Tim Graham

Patch needs improvement: set

You should be able to trigger the Oracle build yourself as described on wiki:Jenkins. I ran the tests locally and put the failures in a PR comment.

comment:12 Changed 10 months ago by Johannes Hoppe

Patch needs improvement: unset

comment:13 Changed 6 months ago by Carlton Gibson

Needs tests: set

comment:14 Changed 6 months ago by Johannes Hoppe

Needs tests: unset

comment:15 Changed 5 months ago by felixxm

Patch needs improvement: set

comment:16 Changed 5 months ago by Johannes Hoppe

Patch needs improvement: unset

comment:17 Changed 5 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In bc91f27a:

Refs #29444 -- Added support for fetching a returned non-integer insert values on Oracle.

This is currently not actively used, since the ORM will ask the
SQL compiler to only return auto fields.

comment:18 Changed 5 months ago by felixxm

Patch needs improvement: set

comment:19 Changed 5 months ago by Johannes Hoppe

Patch needs improvement: unset

comment:20 Changed 3 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In 003bb34b:

Refs #29444 -- Made db.backends.oracle.utils.InsertVar use str as default.

comment:21 Changed 3 months ago by felixxm

Triage Stage: AcceptedReady for checkin

comment:22 Changed 3 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In 736e7d44:

Refs #29444 -- Fixed DateField constructor in db.backends.oracle.utils.InsertVar.

comment:23 Changed 3 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In 7254f113:

Refs #29444 -- Allowed returning multiple fields from INSERT statements on PostgreSQL.

Thanks Florian Apolloner, Tim Graham, Simon Charette, Nick Pope, and
Mariusz Felisiak for reviews.

Changed 3 months ago by felixxm

Attachment: refs-29444-oracle-part.diff added

comment:24 Changed 3 months ago by felixxm

Has patch: unset
Triage Stage: Ready for checkinAccepted

This ticket is doable on Oracle with cx_Oracle 6.3+ (see cx_oracle-63-supports-dml-returning-for-batch-statement) that's why I'm leaving it open. First attempt of fixing this on Oracle was a part of the original PR (see a diff).

Version 0, edited 3 months ago by felixxm (next)

comment:25 Changed 3 months ago by felixxm

Has patch: set
Patch needs improvement: set

comment:26 Changed 3 months ago by Johannes Hoppe

Patch needs improvement: unset

comment:27 Changed 3 months ago by Mariusz Felisiak <felisiak.mariusz@…>

Resolution: fixed
Status: assignedclosed

In b31e6387:

Fixed #29444 -- Allowed returning multiple fields from INSERT statements on Oracle.

comment:28 Changed 3 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In 0110436:

Refs #29444 -- Removed redundant DatabaseFeatures.can_return_multiple_columns_from_insert.

Unnecessary since b31e63879eb5d9717e9f890401f7222e4f00c910.

Note: See TracTickets for help on using tickets.
Back to Top