Opened 6 months ago
Last modified 5 months ago
#35675 new Cleanup/optimization
Reduce impact of parsing crafted templates with repeat tags
Reported by: | Jake Howard | Owned by: | |
---|---|---|---|
Component: | Template system | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Accepted | |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Pull Requests: | How to create a pull request | ||
Description ¶
The template system uses a regex to extract template tags from text. Given certain inputs, this can take an excessive amount of time:
In [2]: %timeit Template("{%" * 2000) 34.7 ms ± 153 µs per loop (mean ± std. dev. of 7 runs, 10 loops each) In [3]: %timeit Template("{%" * 10000) 877 ms ± 1.49 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) In [4]: %timeit Template("{%" * 20000) 3.49 s ± 47 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) In [5]: %timeit Template("{%") 11.5 µs ± 55.3 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)
The cause is excessive backtracking in the pattern used. Since the template system is so versatile and performance-critical, fixing the issue appears non-trivial.
Note: This bug was raised with the Security Team prior to opening, however was not deemed a security vulnerability since parsing untrusted (or semi-trusted) templates is explicitly warned against.
According to the ticket's flags, the next step(s) to move this issue forward are:
- To provide a patch by sending a pull request. Claim the ticket when you start working so that someone else doesn't duplicate effort. Before sending a pull request, review your work against the patch review checklist. Check the "Has patch" flag on the ticket after sending a pull request and include a link to the pull request in the ticket comment when making that update. The usual format is:
[https://github.com/django/django/pull/#### PR]
.
Change History (3)
comment:1 by , 6 months ago
Triage Stage: | Unreviewed → Accepted |
---|---|
Type: | Bug → Cleanup/optimization |
Version: | → dev |
comment:2 by , 5 months ago
Can't we use precompiling a regex pattern to enhance performance by avoiding repeated compilation overhead for frequent matches?
comment:3 by , 5 months ago
Can't we use precompiling a regex pattern
No, this regex is already compiled before execution (see the linked code above). Repeat compiling isn't the performance hit here, it's the backtracking in the pattern itself.
Thank you Jake for taking the time to create this report. Accepting following the conversation within the Security Team.