Opened 5 years ago

Closed 3 years ago

Last modified 3 years ago

#13869 closed New feature (fixed)

Documentation should mention that QuerySet.iterator() method doesn't affect db driver

Reported by: jtiai Owned by: nobody
Component: Documentation Version: 1.2
Severity: Normal Keywords: orm large dataset psycopg2
Cc: timograham@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Some Python database drivers, like psycopg2 does caching if using client side cursors (instantiated with connection.cursor() call).

Driver itself will still cache all data from queryset causing in huge datasets massive usage of memory even Django documentation suggests that using QuerySet.iterator() doesn't cache anything. It should be noted that database drivers still might do that.

Side-effects: This also will lead to double caching in normal situations. First psycopg2 caches raw queryset itself and after that Django caches queryset.

Attachments (1)

13869.diff (798 bytes) - added by timo 3 years ago.

Download all attachments as: .zip

Change History (9)

comment:1 Changed 5 years ago by gabrielhurley

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

comment:3 Changed 4 years ago by graham_king

  • Severity set to Normal
  • Type set to New feature

comment:4 Changed 4 years ago by aaugustin

  • UI/UX unset

Change UI/UX from NULL to False.

comment:5 Changed 4 years ago by aaugustin

  • Easy pickings unset

Change Easy pickings from NULL to False.

Changed 3 years ago by timo

comment:6 Changed 3 years ago by timo

  • Cc timograham@… added
  • Has patch set

comment:7 Changed 3 years ago by Tim Graham <timograham@…>

  • Resolution set to fixed
  • Status changed from new to closed

In 2f722d9728c1946d5d800b3e0b24de7f566a755d:

Fixed #13869 - Warned that QuerySet.iterator() doesn't affect DB driver caching; thanks jtiai for the suggestion.

comment:8 Changed 3 years ago by Tim Graham <timograham@…>

In 6c1c490f64bb5114570a45a523bec365f89f681d:

[1.4.X] Fixed #13869 - Warned that QuerySet.iterator() doesn't affect DB driver caching; thanks jtiai for the suggestion.

Backport of 2f722d9728 from master

Note: See TracTickets for help on using tickets.
Back to Top