Opened 8 hours ago

Last modified 3 hours ago

#36705 assigned Cleanup/optimization

Repeated string concatenation is slow on some python implementations, in particular PyPy

Reported by: Jacob Walls Owned by: Varun Kasyap Pentamaraju
Component: Utilities Version: dev
Severity: Normal Keywords: PyPy
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

There is a known limitation in PyPy that repeated string concatenation is slow. The discussion there observes that CPython has an optimization, but it's fragile and best not relied on.

Seokchan Yoon (thanks for the report!) raised this with the security team, and we decided any hardening here could be done in public.

To close this ticket, we should:

  • implement targeted optimizations in django.utils.numberformat.format() to avoid repeated string concatenation
  • audit for other similar cases
  • Add a disclaimer to our documentation around PyPy, for example, wording like:

That said, a lot of a web framework's work is done by concatenating strings, and PyPy has an issue with that (see https://pypy.org/posts/2023/01/string-concatenation-quadratic.html). This may cause performance issues, depending on your use.

Change History (4)

comment:1 by Natalia Bidart, 6 hours ago

Triage Stage: UnreviewedAccepted

Thank you Jacob!

comment:2 by Varun Kasyap Pentamaraju, 6 hours ago

Owner: set to Varun Kasyap Pentamaraju
Status: newassigned

I am willing to contribute

comment:3 by Varun Kasyap Pentamaraju, 4 hours ago

Has patch: set

comment:4 by Jacob Walls, 3 hours ago

Patch needs improvement: set
Note: See TracTickets for help on using tickets.
Back to Top