Opened 8 hours ago
Last modified 3 hours ago
#36705 assigned Cleanup/optimization
Repeated string concatenation is slow on some python implementations, in particular PyPy
| Reported by: | Jacob Walls | Owned by: | Varun Kasyap Pentamaraju |
|---|---|---|---|
| Component: | Utilities | Version: | dev |
| Severity: | Normal | Keywords: | PyPy |
| Cc: | Triage Stage: | Accepted | |
| Has patch: | yes | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | yes |
| Easy pickings: | no | UI/UX: | no |
Description
There is a known limitation in PyPy that repeated string concatenation is slow. The discussion there observes that CPython has an optimization, but it's fragile and best not relied on.
Seokchan Yoon (thanks for the report!) raised this with the security team, and we decided any hardening here could be done in public.
To close this ticket, we should:
- implement targeted optimizations in django.utils.numberformat.format() to avoid repeated string concatenation
- audit for other similar cases
- Add a disclaimer to our documentation around PyPy, for example, wording like:
That said, a lot of a web framework's work is done by concatenating strings, and PyPy has an issue with that (see https://pypy.org/posts/2023/01/string-concatenation-quadratic.html). This may cause performance issues, depending on your use.
Change History (4)
comment:1 by , 6 hours ago
| Triage Stage: | Unreviewed → Accepted |
|---|
comment:4 by , 3 hours ago
| Patch needs improvement: | set |
|---|
Thank you Jacob!