Opened 6 years ago

Last modified 2 years ago

#12826 new Cleanup/optimization

Use __slots__ for some simple objects.

Reported by: sebastian_noack Owned by: nobody
Component: Core (Other) Version:
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

I have started to introduce slots for some simple object. See attached patch. There are probably more places where slots would make sense. But I have started with django.utils.datastructures.* and django.utils.tree.Node and subclasses of them. Slots make object creation faster and use less memory than object dicts. They are used to improve performance on a lot of python's builtin types and can be used in own code for simple objects.

Attachments (2)

introduce_slots.patch (4.4 KB) - added by sebastian_noack 6 years ago.
benchmark.py (1.5 KB) - added by sebastian_noack 6 years ago.

Download all attachments as: .zip

Change History (10)

Changed 6 years ago by sebastian_noack

comment:1 Changed 6 years ago by Alex

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Design decision needed

Marking as DDN. slots is a bit of a hack, IMO, and for alternate implementations of Python (PyPy) it can degrade performance (although in Unladen Swallow they are an even bigger speed win than CPython).

comment:2 Changed 6 years ago by sebastian_noack

I agree that __slots__ might be confusing to the user and restrict the flexibility of python. But for simple objects as those affected by my patch, this is no problem, because of the way they are used is not a problem with __slots__. So user's of django do not have to take care about it. But they care about django's performance. And other frameworks (e.g. jinja2) are using __slots__ for some classes to improve performance.

However, I have written a benchmark (see attachment). The result below, shows that for PyPy there is even a bigger speed win as for CPython, when using __slots__. I have done the benchmark on version 2.4 (which is equivalent to PyPy 1.0) and 2.6 of CPython. The speed gain for Python 2.4 and 2.6 is similar. For CPython there is no difference in speed when the attributes defined by __slots__ are not set. But usually they are set in the __init__ method and in this case, classes using __slots__ are always faster. But note, if __slots__ is defined the object's __dict__ is not created and in addition to the speed gain the main advantage is that it needs less memory.

$ pypy --version
2.4.1 (pypy 1.0.0 build 55235)

$ pypy benchmark.py
                    | Avg.  | Min.  | Max. 
--------------------+-------+-------+------
WithSlots           | 0.136 | 0.116 | 0.152
--------------------+-------+-------+------
WithSlotsAndCtor    | 0.220 | 0.215 | 0.229
--------------------+-------+-------+------
WithoutSlots        | 0.153 | 0.142 | 0.176
--------------------+-------+-------+------
WithoutSlotsAndCtor | 0.287 | 0.276 | 0.304

$ python2.4 benchmark.py
                    | Avg.  | Min.  | Max. 
--------------------+-------+-------+------
WithSlots           | 0.021 | 0.018 | 0.025
--------------------+-------+-------+------
WithSlotsAndCtor    | 0.093 | 0.090 | 0.101
--------------------+-------+-------+------
WithoutSlots        | 0.021 | 0.020 | 0.021
--------------------+-------+-------+------
WithoutSlotsAndCtor | 0.107 | 0.102 | 0.113

$ python2.6 benchmark.py
                    | Avg.  | Min.  | Max. 
--------------------+-------+-------+------
WithSlots           | 0.026 | 0.024 | 0.028
--------------------+-------+-------+------
WithSlotsAndCtor    | 0.092 | 0.089 | 0.093
--------------------+-------+-------+------
WithoutSlots        | 0.026 | 0.025 | 0.028
--------------------+-------+-------+------
WithoutSlotsAndCtor | 0.113 | 0.107 | 0.125

Changed 6 years ago by sebastian_noack

comment:3 Changed 5 years ago by mtredinnick

  • Triage Stage changed from Design decision needed to Accepted

Slots aren't really a hack, so much as a targeted optimisation for certain cases. They definitely aren't to be used indiscriminately, but can show benefits in small objects that are frequently created and never monkey-patched. The general direction of this ticket is worthwhile. I haven't seriously reviewed the implementation yet.

comment:4 Changed 4 years ago by lukeplant

  • Type set to Cleanup/optimization

comment:5 Changed 4 years ago by lukeplant

  • Severity set to Normal

comment:6 Changed 4 years ago by patchhammer

  • Easy pickings unset
  • Patch needs improvement set

introduce_slots.patch fails to apply cleanly on to trunk

comment:7 Changed 4 years ago by aaugustin

  • UI/UX unset

Change UI/UX from NULL to False.

comment:8 Changed 2 years ago by aaugustin

  • Component changed from Uncategorized to Core (Other)
Note: See TracTickets for help on using tickets.
Back to Top