Opened 5 years ago

Closed 5 years ago

Last modified 5 years ago

#17634 closed Cleanup/optimization (fixed)

MultiValueDict.appendlist is ineffective

Reported by: Miloslav Pojman Owned by: nobody
Component: Core (Other) Version: 1.3
Severity: Normal Keywords:
Cc: miloslav.pojman@…, charette.s@… Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description

Current MultiValueDict.appendlist implementation uses + operator in order to merge two lists -- which is pretty ineffective.

When I replaced a list merge by a simple append call I got aprox. 20% execution time of the original implementation:

    def appendlist(self, key, value):
        super(MultiValueDict, self).setdefault(key, []).append(value)

Many calls to appendlist method are possible if the MultiValueDict is being built incrementally (for example when reading response from a remote API).

Attachments (2)

master...ticket-17634-multivalue-dict-optimisation.diff (592 bytes) - added by Simon Charette 5 years ago.
master...ticket-17634-multivalue-dict-optimisation.diff.1 (1.3 KB) - added by Simon Charette 5 years ago.
With tests from claudep

Download all attachments as: .zip

Change History (11)

comment:1 Changed 5 years ago by Miloslav Pojman

Cc: miloslav.pojman@… added
Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset

comment:2 Changed 5 years ago by Carl Meyer

Component: UncategorizedCore (Other)
Triage Stage: UnreviewedAccepted

Changed 5 years ago by Simon Charette

comment:3 Changed 5 years ago by Simon Charette

Cc: charette.s@… added
Has patch: set

Added the diff which is also reachable on github.

All tests passing using test_sqlite settings with this patch applied on r17419.

comment:4 Changed 5 years ago by Claude Paroz

Needs tests: set

I suggest to add a test specific to appendlist in regressiontests/utils/datastructures.py (class MultiValueDictTests).

comment:5 Changed 5 years ago by Claude Paroz

djangobench results on MultiValueDict.appendlist with current patch:

Min: 0.000008 -> 0.000004: 2.0625x faster
Avg: 0.000008 -> 0.000005: 1.8077x faster
Significant (t=48.456398)

Changed 5 years ago by Simon Charette

With tests from claudep

comment:6 Changed 5 years ago by Simon Charette

Easy pickings: set
Needs tests: unset

Added a patch with test provided by claudep.

comment:7 Changed 5 years ago by Claude Paroz

Triage Stage: AcceptedReady for checkin

Thanks, looks fine now.

comment:8 Changed 5 years ago by Aymeric Augustin

Resolution: fixed
Status: newclosed

In [17464]:

(The changeset message doesn't reference this ticket)

comment:7 Changed 5 years ago by Aymeric Augustin

In [17807]:

[1.3.X] Fixed #17634 -- Optimized the performance of MultiValueDict by using append instead of copy and by minimizing the number of dict lookups. Backport of r17464 from trunk.

Note: See TracTickets for help on using tickets.
Back to Top