Opened 2 months ago

Closed 4 weeks ago

Last modified 4 weeks ago

#36490 closed Cleanup/optimization (fixed)

Avoid unnecessary transaction in QuerySet.bulk_create

Reported by: Simon Charette Owned by: Simon Charette
Component: Database layer (models, ORM) Version: 5.2
Severity: Normal Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

The QuerySet.bulk_create method creates an unnecessary transaction when it can perform the insert in a singe INSERT when the following criteria are met

  1. There's no mix of objects with and without primary key (this could potentially be optimized to be done in a single query but that's another can of worm)
  2. The objects fit in a single batch with respect to the provided batch_size and connection.ops.bulk_batch_size

This is unnecessary overhead we've optimized in other areas of the code base (e.g. Model.save) which seems warranted in this case particularly because bulk_create offers a more featureful interface than create which can be used to insert a small number of rows.

Change History (5)

comment:1 by David Sanders, 2 months ago

Triage Stage: UnreviewedAccepted

comment:2 by Simon Charette, 2 months ago

Has patch: set

comment:3 by Sarah Boyce, 4 weeks ago

Triage Stage: AcceptedReady for checkin

comment:4 by Sarah Boyce <42296566+sarahboyce@…>, 4 weeks ago

Resolution: fixed
Status: assignedclosed

In e1671278:

Fixed #36490 -- Avoided unnecessary transaction in bulk_create.

When dealing with an heterogeneous set of object with regards to primary key
assignment that fits in a single batch there's no need to wrap the single
INSERT statement in a transaction.

comment:5 by Sarah Boyce <42296566+sarahboyce@…>, 4 weeks ago

In 5eca562a:

Refs #36490 -- Simplified QuerySet._batched_insert returning fields handling.

Whether or not returning_fields should be specified to _insert is not a
function of each batches so the conditional can be moved outside of the loop.

Note: See TracTickets for help on using tickets.
Back to Top