Opened 3 days ago

Last modified 3 days ago

#36215 new Cleanup/optimization

Use PEP 448 unpacking to concatenate iterables

Reported by: Aarni Koskela Owned by:
Component: Uncategorized Version: 5.1
Severity: Normal Keywords:
Cc: Nick Pope, Tim Graham, Mariusz Felisiak Triage Stage: Unreviewed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no
Pull Requests:19210 build:success

Description

https://peps.python.org/pep-0448/ (shipped in Python 3.5) adds support for unpacking in tuple/list displays.

This is faster in micro-benchmarks (every little helps), more concise, arguably more readable, and avoids certain classes of bugs where users pass in tuples where Django expects lists and vice versa (see e.g. https://github.com/django/django/pull/15270).

Some instances of this can be automatically found and fixed with the Ruff linter's RUF005 rule (https://docs.astral.sh/ruff/rules/collection-literal-concatenation/).

According to the ticket's flags, the next step(s) to move this issue forward are:

  • For bugs: reproduce the bug. If it's a regression, bisect to find the commit where the behavior changed.
  • For new features or cleanups: give a second opinion of the proposal.
  • In either case, mark the Triage Stage as "Accepted" if the issue seems valid, ask for additional clarification from the reporter, or close the ticket.

Change History (2)

comment:1 by Sarah Boyce, 3 days ago

Cc: Nick Pope Tim Graham Mariusz Felisiak added

I don't love micro-optimization tickets, especially without benchmarks, as it feels like unnecessary churn.
If we are to do this, I think it makes sense to add a point in the Python style guide. Possibly even add a linter.

I see we had a ticket for this previously (#28909).
I am going to cc some folks who were involved in the original ticket in case they have an opinion.

comment:2 by Aarni Koskela, 3 days ago

Ah, sorry, yeah, I left the microbenchmarks out of the original ticket text for brevity.

BLANK_CHOICE_DASH = [("", "---------")]

def get_action_choices_1(default_choices=BLANK_CHOICE_DASH):
    choices = [] + default_choices
    ...
    return choices


def get_action_choices_2(default_choices=BLANK_CHOICE_DASH):
    choices = [*default_choices]
    ...
    return choices

(adapted from what get_action_choices in the admin does)

results in

$ uv run --python=3.13 concates.py
name='get_action_choices_1' iters=5000000 time=0.247 iters_per_sec=20206826.98
name='get_action_choices_2' iters=5000000 time=0.207 iters_per_sec=24111147.61
$ uv run --python=3.12 concates.py
name='get_action_choices_1' iters=5000000 time=0.333 iters_per_sec=15023921.85
name='get_action_choices_2' iters=5000000 time=0.220 iters_per_sec=22699169.61

However, now that I benchmark some other expressions (such as some tuple concatenations), they may actually be slower in some of these cases; it looks like CPython internally converts the first input tuple to a list (BUILD_LIST), then calls an INTRINSIC_LIST_TO_TUPLE opcode on it...

Note: See TracTickets for help on using tickets.
Back to Top