Opened 6 years ago

Closed 2 years ago

Last modified 17 months ago

#29984 closed New feature (fixed)

Support prefetch_related() with Queryset.iterator()

Reported by: Taylor H Owned by: Jacob Walls
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Asif Saifuddin Auvi)

I was surprised when I found out that prefetch_related calls are ignored when using Queryset.iterator. I noticed in the docs here https://docs.djangoproject.com/en/dev/ref/models/querysets/#iterator that it is because the "these two optimizations do not make sense together." That may have been true in the past, but it is definitely not the case now. The iterator allows chunking (by default 2000) and it would be very helpful to prefetch related for each chunk.

Change History (12)

comment:1 by Tim Graham, 6 years ago

Summary: Support prefetch_related with Queryset.iteratorSupport prefetch_related() with Queryset.iterator()
Triage Stage: UnreviewedAccepted

The situation isn't completely straightforward. There's a discussion on django-developers. Accepting the ticket to do something.

comment:2 by Raphael Kimmig, 6 years ago

I've created a proof of concept implementation https://github.com/django/django/pull/10707/

I think prefetching is probably closer to what a user would expect from the api.

comment:3 by David Foster, 5 years ago

Has patch: set
Needs documentation: set

comment:4 by Asif Saifuddin Auvi, 5 years ago

Description: modified (diff)
Version: 2.1master

comment:5 by Simon Charette, 4 years ago

Patch needs improvement: set

Patch needs both documentation and a deprecation warning when self._prefetch_related_lookups and chunk_size is None. The idea is to eventually turn this deprecation warning into an error and never performs prefetching unless a chunk size is specified.

comment:6 by Jacob Walls, 2 years ago

Owner: changed from nobody to Jacob Walls
Status: newassigned

comment:7 by Jacob Walls, 2 years ago

Needs documentation: unset
Patch needs improvement: unset

comment:8 by Mariusz Felisiak, 2 years ago

Patch needs improvement: set

comment:9 by Jacob Walls, 2 years ago

Patch needs improvement: unset

comment:10 by Mariusz Felisiak, 2 years ago

Triage Stage: AcceptedReady for checkin

comment:11 by Mariusz Felisiak <felisiak.mariusz@…>, 2 years ago

Resolution: fixed
Status: assignedclosed

In edbf9302:

Fixed #29984 -- Added QuerySet.iterator() support for prefetching related objects.

Co-authored-by: Raphael Kimmig <raphael.kimmig@…>
Co-authored-by: Simon Charette <charette.s@…>

comment:12 by Mariusz Felisiak <felisiak.mariusz@…>, 17 months ago

In 1391356:

Refs #29984 -- Made QuerySet.iterator() without chunk_size raise ValueError after prefetch_related().

Per deprecation timeline.

Note: See TracTickets for help on using tickets.
Back to Top