Opened 21 months ago
Closed 20 months ago
#35250 closed Cleanup/optimization (fixed)
Stop URL system checks from compiling regular expressions
| Reported by: | Adam Johnson | Owned by: | Adam Johnson | 
|---|---|---|---|
| Component: | Core (System checks) | Version: | dev | 
| Severity: | Normal | Keywords: | |
| Cc: | Triage Stage: | Ready for checkin | |
| Has patch: | yes | Needs documentation: | no | 
| Needs tests: | no | Patch needs improvement: | no | 
| Easy pickings: | no | UI/UX: | no | 
Description (last modified by )
Continuing my project to optimize the system checks, I found some good optimizations under  django.core.checks.urls.check_url_config(), which showed up as quite expensive in profiling.
Looking down the call tree, it seems the most expensive part of this process is compiling the each URL pattern’s regular expression. This is unnecessary work though, as the checks only need *uncompiled* regular expression patterns. Using the compiled versions “undoes” the lazy-compile optimization that LocaleRegexDescriptor was created for in #27453 / 6e222dae5636f875c19ec66f730a4241abe33faa, at least for any process that runs checks.
The checks were fetching the uncompiled pattern with self.regex.pattern, which makse LocaleRegexDescriptor  compile the pattern only to then read the uncompiled pattern from its pattern attribute.
Additionally, RoutePattern was calling _route_to_regex() twice to fetch its two result variables in different places: once in __init__() and again in _compile() (in the non-translated case). This function has non-trivial cost so avoiding double execution is worth it.
Before optimization stats:
check_url_configtook 67ms, or ~10% of the time for checks.LocaleRegexDescriptor.__get__()showed 965 calls taking ~60ms, ~9% of the total runtime of checks.re.compile()showed 741 calls for 94ms._route_to_regex()had 1900 calls taking 18ms (~2.6% of the total runtime).
After optimization:
check_url_config()took 5ms, ~0.9% of the new total runtime.- The calls to 
LocaleRegexDescriptor.__get__are gone. re.compile()drops to 212 calls from other sites, for a total of 51ms._route_to_regex()drops to the expected 950 calls, taking half the time at 9ms.
(I also tried benchmarking with django-asv but got inconclusive results where change was within the error margins.)
Change History (8)
comment:1 by , 21 months ago
| Description: | modified (diff) | 
|---|
comment:2 by , 21 months ago
| Description: | modified (diff) | 
|---|
comment:3 by , 21 months ago
| Triage Stage: | Unreviewed → Accepted | 
|---|
comment:4 by , 21 months ago
| Patch needs improvement: | set | 
|---|
comment:5 by , 20 months ago
| Patch needs improvement: | unset | 
|---|
comment:6 by , 20 months ago
| Triage Stage: | Accepted → Ready for checkin | 
|---|
Tentatively accepted.