Opened 6 years ago

Closed 6 years ago

Last modified 4 years ago

#12389 closed (fixed)

performance enhancement for testsuite

Reported by: Travis Cline <travis.cline@…> Owned by: nobody
Component: Core (Other) Version: master
Severity: Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:


In my profiling of the test suite get_models appears to be the lowest hanging fruit. Attached is a patch that gives about a 30-35% speed increase for the django testsuite and closer to 40% for Ellington's suite.

Attachments (1)

get_models_caching.diff (2.0 KB) - added by Travis Cline <travis.cline@…> 6 years ago.

Download all attachments as: .zip

Change History (7)

Changed 6 years ago by Travis Cline <travis.cline@…>

comment:1 Changed 6 years ago by Alex

  • Component changed from Testing framework to Core framework
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Ready for checkin

Though this patch helps testing I've changed the component since the change is actually in DB loading.

comment:2 Changed 6 years ago by lukeplant

  • Resolution set to fixed
  • Status changed from new to closed

(In [11883]) Fixed #12389 - performance enhancement of get_models()

Thanks to Travis Cline for the patch, which gives about 35% speed increase
for the testsuite (with SQLite in-memory DB).

comment:3 Changed 6 years ago by lukeplant

FYI, I tweaked the patch in a few ways, in the spirit of micro-optimisation:

  1. Used try/except to avoid doing two dictionary lookups.
  2. Moved cache check before self._populate() call.
  3. Used a tuple as the cache key, rather than a string, mainly for simplicity and clarity.

Only the last of these had any measurable effect at all, and not much, but they made me feel better :-)

comment:4 Changed 6 years ago by Alex

Using a tuple helped? That's suprising, CPython dictionaries have some specific optimizations for when strings are used.

comment:5 Changed 6 years ago by Travis Cline <travis.cline@…>

I did some timeit benchmarks and it is quicker -- because of the skipping of the creation of the string -- if you already have the string the string lookup is about twice as fast as the tuple lookup (both super quick though).

comment:6 Changed 4 years ago by jacob

  • milestone 1.2 deleted

Milestone 1.2 deleted

Note: See TracTickets for help on using tickets.
Back to Top