Opened 6 years ago

Closed 4 years ago

Last modified 4 years ago

#13869 closed New feature (fixed)

Documentation should mention that QuerySet.iterator() method doesn't affect db driver

Reported by: Jani Tiainen Owned by: nobody
Component: Documentation Version: 1.2
Severity: Normal Keywords: orm large dataset psycopg2
Cc: timograham@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Some Python database drivers, like psycopg2 does caching if using client side cursors (instantiated with connection.cursor() call).

Driver itself will still cache all data from queryset causing in huge datasets massive usage of memory even Django documentation suggests that using QuerySet.iterator() doesn't cache anything. It should be noted that database drivers still might do that.

Side-effects: This also will lead to double caching in normal situations. First psycopg2 caches raw queryset itself and after that Django caches queryset.

Attachments (1)

13869.diff (798 bytes) - added by Tim Graham 4 years ago.

Download all attachments as: .zip

Change History (9)

comment:1 Changed 6 years ago by Gabriel Hurley

Triage Stage: UnreviewedAccepted

comment:3 Changed 6 years ago by Graham King

Severity: Normal
Type: New feature

comment:4 Changed 5 years ago by Aymeric Augustin

UI/UX: unset

Change UI/UX from NULL to False.

comment:5 Changed 5 years ago by Aymeric Augustin

Easy pickings: unset

Change Easy pickings from NULL to False.

Changed 4 years ago by Tim Graham

Attachment: 13869.diff added

comment:6 Changed 4 years ago by Tim Graham

Cc: timograham@… added
Has patch: set

comment:7 Changed 4 years ago by Tim Graham <timograham@…>

Resolution: fixed
Status: newclosed

In 2f722d9728c1946d5d800b3e0b24de7f566a755d:

Fixed #13869 - Warned that QuerySet.iterator() doesn't affect DB driver caching; thanks jtiai for the suggestion.

comment:8 Changed 4 years ago by Tim Graham <timograham@…>

In 6c1c490f64bb5114570a45a523bec365f89f681d:

[1.4.X] Fixed #13869 - Warned that QuerySet.iterator() doesn't affect DB driver caching; thanks jtiai for the suggestion.

Backport of 2f722d9728 from master

Note: See TracTickets for help on using tickets.
Back to Top