Opened 2 years ago

Closed 12 months ago

Last modified 12 months ago

#29444 closed Cleanup/optimization (fixed)

Allow fields to be part of the RETURNING clause during INSERT

Reported by: Johannes Hoppe Owned by: Johannes Hoppe
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords: django, db, returning, default, model, field
Cc: info@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Dependency for #27452

The SQL RETURNING statement is currently used for inserts of new objects. Mainly it is responsible to return the primary key in a single query.

That's great but currently in the postgres database backend this value is hard coded to only the primary key. I would like to propose to make this configurable.

There are plenty of uses cases for such a feature ultimately even enabling database defaults, but I would suggest to keep it very simple for now.

I would suggest to simply allow model fields to specify if they should be included in the returning statement or not. The default being not (except PK) to maintain the current behavior.

The implementation as well as the related code lines have been already discussed here:
https://github.com/django/django/pull/7525#issuecomment-354269077

I highly suggest to keep this a Postgres ONLY feature for now. That way we can see what sticks and consider extending this functionality into django.db in the future after we have seen how it's being used and gathered feedback.

Attachments (1)

refs-29444-oracle-part.diff (8.2 KB) - added by felixxm 13 months ago.

Download all attachments as: .zip

Change History (29)

comment:1 Changed 2 years ago by Johannes Hoppe

Version: 2.0master

comment:2 Changed 2 years ago by Johannes Hoppe

Owner: set to Johannes Hoppe
Status: newassigned

comment:3 Changed 2 years ago by Johannes Hoppe

Has patch: set
Last edited 2 years ago by Tim Graham (previous) (diff)

comment:4 Changed 2 years ago by Johannes Hoppe

Patch needs improvement: set
Triage Stage: UnreviewedAccepted

comment:5 Changed 2 years ago by Johannes Hoppe

Patch needs improvement: unset

comment:6 Changed 2 years ago by Tim Graham

Patch needs improvement: set

comment:7 Changed 2 years ago by Johannes Hoppe

Patch needs improvement: unset
Type: New featureCleanup/optimization

comment:8 Changed 2 years ago by Tim Graham

Component: contrib.postgresDatabase layer (models, ORM)
Patch needs improvement: set
Summary: Allow modification of RETURNING values in django.contrib.postgresAllow fields to be part of the RETURNING clause during INSERT

A number of Oracle test failures remain.

comment:9 Changed 21 months ago by Johannes Hoppe

Patch needs improvement: unset

I fixed a couple of things. Nothing fails for me now locally that doesn't fail on master. Can you trigger it again?

comment:10 Changed 20 months ago by Tim Graham <timograham@…>

In b131f9c7:

Refs #29444 -- Renamed DatabaseFeatures.can_return_id* to be generic for other columns.

comment:11 Changed 20 months ago by Tim Graham

Patch needs improvement: set

You should be able to trigger the Oracle build yourself as described on wiki:Jenkins. I ran the tests locally and put the failures in a PR comment.

comment:12 Changed 19 months ago by Johannes Hoppe

Patch needs improvement: unset

comment:13 Changed 16 months ago by Carlton Gibson

Needs tests: set

comment:14 Changed 16 months ago by Johannes Hoppe

Needs tests: unset

comment:15 Changed 15 months ago by felixxm

Patch needs improvement: set

comment:16 Changed 15 months ago by Johannes Hoppe

Patch needs improvement: unset

comment:17 Changed 15 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In bc91f27a:

Refs #29444 -- Added support for fetching a returned non-integer insert values on Oracle.

This is currently not actively used, since the ORM will ask the
SQL compiler to only return auto fields.

comment:18 Changed 15 months ago by felixxm

Patch needs improvement: set

comment:19 Changed 15 months ago by Johannes Hoppe

Patch needs improvement: unset

comment:20 Changed 13 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In 003bb34b:

Refs #29444 -- Made db.backends.oracle.utils.InsertVar use str as default.

comment:21 Changed 13 months ago by felixxm

Triage Stage: AcceptedReady for checkin

comment:22 Changed 13 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In 736e7d44:

Refs #29444 -- Fixed DateField constructor in db.backends.oracle.utils.InsertVar.

comment:23 Changed 13 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In 7254f113:

Refs #29444 -- Allowed returning multiple fields from INSERT statements on PostgreSQL.

Thanks Florian Apolloner, Tim Graham, Simon Charette, Nick Pope, and
Mariusz Felisiak for reviews.

Changed 13 months ago by felixxm

Attachment: refs-29444-oracle-part.diff added

comment:24 Changed 13 months ago by felixxm

Has patch: unset
Triage Stage: Ready for checkinAccepted

This ticket is doable on Oracle with cx_Oracle 6.3+ (see cx_oracle-63-supports-dml-returning-for-batch-statement) that's why I'm leaving it open. First attempt of fixing this on Oracle was a part of the original PR (see a diff).

Version 0, edited 13 months ago by felixxm (next)

comment:25 Changed 12 months ago by felixxm

Has patch: set
Patch needs improvement: set

comment:26 Changed 12 months ago by Johannes Hoppe

Patch needs improvement: unset

comment:27 Changed 12 months ago by Mariusz Felisiak <felisiak.mariusz@…>

Resolution: fixed
Status: assignedclosed

In b31e6387:

Fixed #29444 -- Allowed returning multiple fields from INSERT statements on Oracle.

comment:28 Changed 12 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In 0110436:

Refs #29444 -- Removed redundant DatabaseFeatures.can_return_multiple_columns_from_insert.

Unnecessary since b31e63879eb5d9717e9f890401f7222e4f00c910.

Note: See TracTickets for help on using tickets.
Back to Top