#35467 closed Cleanup/optimization (fixed)
Prefer urlsplit() over urlparse()
Reported by: | Adam Johnson | Owned by: | Jake Howard |
---|---|---|---|
Component: | Utilities | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Ready for checkin | |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description (last modified by )
Many places in Django use [urlparse()
](https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urlparse), which supports the rarely-used “path parameter” syntax (not to be confused with query parameters). The urlsplit()
function is similar but does not parse such path parameters, which makes it a bit faster.
I think most or all calls to urlparse()
can be replaced with urlsplit()
, and similarly urlunparse()
with urlunsplit()
. This may make a small but measurable performance difference in common paths, such as in CsrfViewMiddleware
or the test Client
.
See more in this Anthony Sottile video: https://www.youtube.com/watch?v=ABJvdsIANds , where he reports a 3% import time improvement on the Stripe project.
Change History (8)
comment:1 by , 5 months ago
Description: | modified (diff) |
---|
comment:2 by , 5 months ago
Triage Stage: | Unreviewed → Accepted |
---|
comment:3 by , 5 months ago
Owner: | changed from | to
---|---|
Status: | new → assigned |
From some basic testing, it looks like this should have a nice improvement:
In [1]: import urllib.parse In [2]: %timeit urllib.parse.urlparse("https://example.com") 1.52 µs ± 37.9 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each) In [3]: %timeit urllib.parse.urlsplit("https://example.com") 258 ns ± 1.14 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)
The difference stays about constant, even with longer URLs. The specifics will obviously vary by hardware and Python version (the above is 3.12.3), but ~6x improvement is definitely worthwhile.
comment:5 by , 4 months ago
Needs tests: | set |
---|---|
Patch needs improvement: | set |
comment:6 by , 4 months ago
Needs tests: | unset |
---|---|
Patch needs improvement: | unset |
Comments addressed / replied-to in PR.
comment:7 by , 4 months ago
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
comment:8 by , 4 months ago
Triage Stage: | Accepted → Ready for checkin |
---|
Accepting for someone to make updates and confirm with benchmarks. 👍