Use __slots__ for some simple objects.

[Sebastian Noack]

I have started to introduce slots for some simple object. See attached patch. There are probably more places where slots would make sense. But I have started with django.utils.datastructures.* and django.utils.tree.Node and subclasses of them. Slots make object creation faster and use less memory than object dicts. They are used to improve performance on a lot of python's builtin types and can be used in own code for simple objects.

[Alex Gaynor]

Marking as DDN. slots is a bit of a hack, IMO, and for alternate implementations of Python (PyPy) it can degrade performance (although in Unladen Swallow they are an even bigger speed win than CPython).

[Sebastian Noack]

I agree that __slots__ might be confusing to the user and restrict the flexibility of python. But for simple objects as those affected by my patch, this is no problem, because of the way they are used is not a problem with __slots__. So user's of django do not have to take care about it. But they care about django's performance. And other frameworks (e.g. jinja2) are using __slots__ for some classes to improve performance.

However, I have written a benchmark (see attachment). The result below, shows that for PyPy there is even a bigger speed win as for CPython, when using __slots__. I have done the benchmark on version 2.4 (which is equivalent to PyPy 1.0) and 2.6 of CPython. The speed gain for Python 2.4 and 2.6 is similar. For CPython there is no difference in speed when the attributes defined by __slots__ are not set. But usually they are set in the __init__ method and in this case, classes using __slots__ are always faster. But note, if __slots__ is defined the object's __dict__ is not created and in addition to the speed gain the main advantage is that it needs less memory.

$ pypy --version
2.4.1 (pypy 1.0.0 build 55235)

$ pypy benchmark.py
                    | Avg.  | Min.  | Max. 
--------------------+-------+-------+------
WithSlots           | 0.136 | 0.116 | 0.152
--------------------+-------+-------+------
WithSlotsAndCtor    | 0.220 | 0.215 | 0.229
--------------------+-------+-------+------
WithoutSlots        | 0.153 | 0.142 | 0.176
--------------------+-------+-------+------
WithoutSlotsAndCtor | 0.287 | 0.276 | 0.304

$ python2.4 benchmark.py
                    | Avg.  | Min.  | Max. 
--------------------+-------+-------+------
WithSlots           | 0.021 | 0.018 | 0.025
--------------------+-------+-------+------
WithSlotsAndCtor    | 0.093 | 0.090 | 0.101
--------------------+-------+-------+------
WithoutSlots        | 0.021 | 0.020 | 0.021
--------------------+-------+-------+------
WithoutSlotsAndCtor | 0.107 | 0.102 | 0.113

$ python2.6 benchmark.py
                    | Avg.  | Min.  | Max. 
--------------------+-------+-------+------
WithSlots           | 0.026 | 0.024 | 0.028
--------------------+-------+-------+------
WithSlotsAndCtor    | 0.092 | 0.089 | 0.093
--------------------+-------+-------+------
WithoutSlots        | 0.026 | 0.025 | 0.028
--------------------+-------+-------+------
WithoutSlotsAndCtor | 0.113 | 0.107 | 0.125

[Malcolm Tredinnick]

Slots aren't really a hack, so much as a targeted optimisation for certain cases. They definitely aren't to be used indiscriminately, but can show benefits in small objects that are frequently created and never monkey-patched. The general direction of this ticket is worthwhile. I haven't seriously reviewed the implementation yet.

[patchhammer]

introduce_slots.patch fails to apply cleanly on to trunk

[Aymeric Augustin]

Change UI/UX from NULL to False.

[Carlton Gibson]

I think at this latter day a widespread switch to slots isn't actionable. (We'd need a DEP, with an implementation plan and all the rest of it.)
I guess that's why there's been no progress made in many years.

Yes, in theory, there may be gains to be made. As such, individual adjustments to specific classes can be assessed on a case-by-case basis.
Maybe that way we move forward.

I hope that makes sense.