Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#18949 closed Cleanup/optimization (fixed)

model_to_dict is slow for tables with very large fields

Reported by: brian@… Owned by: nobody
Component: Forms Version: 1.4
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description

In ModelForms, when populating initial data using django.forms.models.model_to_dict, tables with very large text fields can slow things down quite a bit. This can be fixed by using the values_list method of a queryset. The current line:

data[f.name] = [obj.pk for obj in f.value_from_object(instance)]

could be written:

data[f.name] = f.value_from_object(instance).values_list('pk', flat=True)

Change History (8)

comment:1 Changed 3 years ago by claudep

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

comment:2 Changed 3 years ago by anonymous

Note that while recently using a similar construct in some code, I discovered that values_list does not actually return a list. Normally this would be a good thing, but in some cases this could cause backward incompatible behavior. It might be preferable to cast the return value to a list before returning it to maintain 100% backward compatibility.

comment:4 Changed 3 years ago by Anton I. Sipos <aisipos@…>

  • Resolution set to fixed
  • Status changed from new to closed

In e44ab5bb4fd3aa826ca4243a8ea9fd7125800da2:

Fixed #18949 -- Improve performance of model_to_dict with many-to-many

When calling model_to_dict, improve performance of the generated SQL by
using values_list to determine primary keys of many to many objects. Add
a specific test for this function, test_model_to_dict_many_to_many

Thanks to brian for the original report and suggested fix.

comment:5 Changed 3 years ago by Alex Gaynor <alex.gaynor@…>

In 4d766b3c9aca36cbe7dc71df0cc93fb6f9deea60:

Merge pull request #495 from aisipos/ticket_18949

Fixed #18949 -- Improve performance of model_to_dict with many-to-many

comment:6 Changed 3 years ago by Alex Gaynor <alex.gaynor@…>

In d828d4e186b5433f637b86c4e44a3b68acab5cb8:

[1.5.x] Merge pull request #495 from aisipos/ticket_18949

Fixed #18949 -- Improve performance of model_to_dict with many-to-many

Backport of 4d766b3c9aca36cbe7dc71df0cc93fb6f9deea60.

comment:7 Changed 3 years ago by Preston Holmes <preston@…>

In 81342c28c6e4fbf98296555691750c33bce1bbc7:

[1.5.x] Fixed #18949 -- Fix broken test interactions in ModelForms tests

A test in Model Forms test was specifically referring to a fixed
primary key, which was now being used up in a newly committed.
This has been worked around by specifying a higher primary
key.

comment:8 Changed 3 years ago by Preston Holmes <preston@…>

In fdea2621cd3f3de472afaab7aa7152a1dc4f505c:

Fixed #18949 -- Fix broken test interactions in ModelForms tests

A test in Model Forms test was specifically referring to a fixed
primary key, which was now being used up in a newly committed.
This has been worked around by specifying a higher primary
key.

Note: See TracTickets for help on using tickets.
Back to Top