Opened 9 years ago

Closed 9 years ago

Last modified 7 years ago

#12389 closed (fixed)

performance enhancement for testsuite

Reported by: Travis Cline <travis.cline@…> Owned by: nobody
Component: Core (Other) Version: master
Severity: Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


In my profiling of the test suite get_models appears to be the lowest hanging fruit. Attached is a patch that gives about a 30-35% speed increase for the django testsuite and closer to 40% for Ellington's suite.

Attachments (1)

get_models_caching.diff (2.0 KB) - added by Travis Cline <travis.cline@…> 9 years ago.

Download all attachments as: .zip

Change History (7)

Changed 9 years ago by Travis Cline <travis.cline@…>

Attachment: get_models_caching.diff added

comment:1 Changed 9 years ago by Alex Gaynor

Component: Testing frameworkCore framework
Triage Stage: UnreviewedReady for checkin

Though this patch helps testing I've changed the component since the change is actually in DB loading.

comment:2 Changed 9 years ago by Luke Plant

Resolution: fixed
Status: newclosed

(In [11883]) Fixed #12389 - performance enhancement of get_models()

Thanks to Travis Cline for the patch, which gives about 35% speed increase
for the testsuite (with SQLite in-memory DB).

comment:3 Changed 9 years ago by Luke Plant

FYI, I tweaked the patch in a few ways, in the spirit of micro-optimisation:

  1. Used try/except to avoid doing two dictionary lookups.
  2. Moved cache check before self._populate() call.
  3. Used a tuple as the cache key, rather than a string, mainly for simplicity and clarity.

Only the last of these had any measurable effect at all, and not much, but they made me feel better :-)

comment:4 Changed 9 years ago by Alex Gaynor

Using a tuple helped? That's suprising, CPython dictionaries have some specific optimizations for when strings are used.

comment:5 Changed 9 years ago by Travis Cline <travis.cline@…>

I did some timeit benchmarks and it is quicker -- because of the skipping of the creation of the string -- if you already have the string the string lookup is about twice as fast as the tuple lookup (both super quick though).

comment:6 Changed 7 years ago by Jacob

milestone: 1.2

Milestone 1.2 deleted

Note: See TracTickets for help on using tickets.
Back to Top