Opened 8 years ago

Closed 7 years ago

#28238 closed Cleanup/optimization (needsinfo)

Changelist save method causing MemoryError

Reported by: kainbozzetto Owned by: nobody
Component: contrib.admin Version: 1.11
Severity: Normal Keywords: admin list_editable post get_queryset memoryerror
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

We are encountering a MemoryError when in the Django admin and saving a changelist of a model that has a large amount of entries. This appears to be due to the size of the queryset returned by get_queryset().

We are currently circumventing this by modifying get_queryset() to check if the request method is a POST and then returning a queryset based on the ids of the form elements.

def get_queryset(self, request):
    if request.method == 'POST':
        return ModelClass.objects.filter(
            id__in=self.get_changelist_ids(request))
    else:
        ...

def get_changelist_ids(self, request):
    ids = []
    n = 0
    while True:
        id = request.POST.get('form-{}-id'.format(n))
        if id:
            ids.append(id)
        else:
            break
        n += 1
    return ids

Our setup has 1GB RAM per process and a model with 100,000+ entries.

Change History (2)

comment:1 by Tim Graham, 7 years ago

Offhand, it's difficult to know if a patch is feasible. Did you plan to offer one?

comment:2 by Tim Graham, 7 years ago

Resolution: needsinfo
Status: newclosed
Type: UncategorizedCleanup/optimization

Closing in absence of a concrete proposal. Feel free to reopen if a patch is provided.

Note: See TracTickets for help on using tickets.
Back to Top