Opened 5 years ago

Closed 20 months ago

Last modified 8 months ago

#29984 closed New feature (fixed)

Support prefetch_related() with Queryset.iterator()

Reported by: Taylor H Owned by: Jacob Walls
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Asif Saifuddin Auvi)

I was surprised when I found out that prefetch_related calls are ignored when using Queryset.iterator. I noticed in the docs here that it is because the "these two optimizations do not make sense together." That may have been true in the past, but it is definitely not the case now. The iterator allows chunking (by default 2000) and it would be very helpful to prefetch related for each chunk.

Change History (12)

comment:1 Changed 5 years ago by Tim Graham

Summary: Support prefetch_related with Queryset.iteratorSupport prefetch_related() with Queryset.iterator()
Triage Stage: UnreviewedAccepted

The situation isn't completely straightforward. There's a discussion on django-developers. Accepting the ticket to do something.

comment:2 Changed 5 years ago by Raphael Kimmig

I've created a proof of concept implementation

I think prefetching is probably closer to what a user would expect from the api.

comment:3 Changed 4 years ago by David Foster

Has patch: set
Needs documentation: set

comment:4 Changed 4 years ago by Asif Saifuddin Auvi

Description: modified (diff)
Version: 2.1master

comment:5 Changed 3 years ago by Simon Charette

Patch needs improvement: set

Patch needs both documentation and a deprecation warning when self._prefetch_related_lookups and chunk_size is None. The idea is to eventually turn this deprecation warning into an error and never performs prefetching unless a chunk size is specified.

comment:6 Changed 21 months ago by Jacob Walls

Owner: changed from nobody to Jacob Walls
Status: newassigned

comment:7 Changed 20 months ago by Jacob Walls

Needs documentation: unset
Patch needs improvement: unset

comment:8 Changed 20 months ago by Mariusz Felisiak

Patch needs improvement: set

comment:9 Changed 20 months ago by Jacob Walls

Patch needs improvement: unset

comment:10 Changed 20 months ago by Mariusz Felisiak

Triage Stage: AcceptedReady for checkin

comment:11 Changed 20 months ago by Mariusz Felisiak <felisiak.mariusz@…>

Resolution: fixed
Status: assignedclosed

In edbf9302:

Fixed #29984 -- Added QuerySet.iterator() support for prefetching related objects.

Co-authored-by: Raphael Kimmig <raphael.kimmig@…>
Co-authored-by: Simon Charette <charette.s@…>

comment:12 Changed 8 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In 1391356:

Refs #29984 -- Made QuerySet.iterator() without chunk_size raise ValueError after prefetch_related().

Per deprecation timeline.

Note: See TracTickets for help on using tickets.
Back to Top