Opened 9 months ago

Last modified 5 weeks ago

#28462 new Bug

ModelAdmin.list_editable unusably slow and memory intensive with large datasets

Reported by: Ben Cole Owned by: nobody
Component: contrib.admin Version: 1.10
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Since 1.10 list_editable on ModelAdmin is unusable for Models with a large-ish dataset.

The problem is caused by a recent change to how the FormSet is generated for the admin. Previously it was generated from the ChangeList result list, but it has been changed to use the admin's get_queryset which will return more than the current "pageful" of results (potentially the entire dataset) causing Django to generate a form for each instance. This results in Django consuming all available RAM and in some cases the python instance crashing. My personal laptop became unresponsive and I had to force power off.

See https://github.com/django/django/commit/917cc288a38f3c114a5440f0749b7e5e1086eb36#commitcomment-23412084

Attachments (1)

memory-problem.png (6.7 KB) - added by Pablo Martín 6 weeks ago.

Download all attachments as: .zip

Change History (7)

comment:1 Changed 9 months ago by Tim Graham

Summary: list_editable unusable with large datasetsModelAdmin.list_editable unusably slow and memory intensive with large datasets
Triage Stage: UnreviewedAccepted

comment:2 Changed 9 months ago by Tim Graham

Has patch: set
Needs tests: set

PR (without a test)

comment:3 Changed 6 weeks ago by Pablo Martín

+1 Same error. We have overwritten the ModelAdmin for this problem. I think to solve a strange problem and getting a memory problem is not a good solution. I prefer a 500 error one time every three month that a serious memory problem every day.

This commit https://github.com/django/django/commit/917cc288a38f3c114a5440f0749b7e5e1086eb36#commitcomment-23412084 does not make any sense.

Four examples of this big problem:


Please consider revert this change, it is very dangerous for other people if they don't identify this bug/trap.

Last edited 6 weeks ago by Pablo Martín (previous) (diff)

Changed 6 weeks ago by Pablo Martín

Attachment: memory-problem.png added

comment:4 Changed 5 weeks ago by Pablo Martín

Easy pickings: set

comment:5 Changed 5 weeks ago by Tim Graham

Easy pickings: unset

I don't think reverting is a good idea as that reintroduces possible data loss.

comment:6 Changed 5 weeks ago by Rubén Gómez

I just made a PR to solve this problem: https://github.com/django/django/pull/9820

Note: See TracTickets for help on using tickets.
Back to Top