Opened 8 months ago

Closed 8 months ago

Last modified 6 months ago

#34822 closed New feature (fixed)

Provide migration serializer for `_lru_cache_wrapper`

Reported by: Natalia Bidart Owned by: Nick Pope
Component: Migrations Version: dev
Severity: Normal Keywords: serializer functools cache
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


Following ticket #24561, and this forum post, it would be useful to have the migrations serializer knowing how to serialize the _lru_cache_wrapper type which is returned when calling the functools.cache and functools.lru_cache decorators.

An example of a use case would be a callable defined for either model field's choices or default that perform an expensive calculation (either CPI or IO bound), and such callable is wrapped with a caching decorator. For instance:

 import functools

    def fib(n):
        if n < 2:
            return n
        return fib(n - 1) + fib(n - 2)

    def get_choices():
        return {i: fib(i) for i in range(20)}

    class Spiral(models.Model):
        size = models.IntegerField(choices=get_choices)

Currently, the error that is raised when generating migrations (assuming support for callable choices), is:

ValueError: Cannot serialize: <functools._lru_cache_wrapper object at 0x7f819ab3b270>

Lastly, while I do appreciate this could be done with a custom serializer, I think it would be of value to provide this in Django core since using the mentioned the decorators is not (IMHO) a niche case.

Change History (9)

comment:1 by Mariusz Felisiak, 8 months ago

Triage Stage: UnreviewedAccepted

Tentatively accepted, let's see how complicated it will be.

comment:3 by Nick Pope, 8 months ago

Has patch: set
Owner: changed from nobody to Nick Pope
Status: newassigned

comment:4 by Natalia Bidart, 8 months ago

Patch needs improvement: set

Settings as patch needs improvement due to minor issues with the docs.

comment:5 by Nick Pope, 8 months ago

Patch needs improvement: unset

comment:6 by Natalia Bidart, 8 months ago

Triage Stage: AcceptedReady for checkin

comment:7 by Natalia <124304+nessita@…>, 8 months ago

In c131949:

Refs #34822 -- Added tests for serializing decorated functions in migrations.

Functions decorated with a decorator that is properly wrapped, e.g. by
using @functools.wraps, are already supported.

comment:8 by Natalia <124304+nessita@…>, 8 months ago

Resolution: fixed
Status: assignedclosed

In f92e68c3:

Fixed #34822 -- Added support for serializing functions decorated with functools.lru_cache in migrations.

@functools.cache and @functools.lru_cache return an object of type
functools._lru_cache_wrapper which prevented the migrations serializer from
working. Simply using the existing FunctionTypeSerializer for this additional
type works as expected.

comment:9 by Mariusz Felisiak <felisiak.mariusz@…>, 6 months ago

In 5f9e5c1:

Refs #34822, Refs #34986 -- Fixed migrations serializer support for functools.lru_cache().

It turns out that functools._lru_cache_wrapper is only a class when
CPython's _functools C module provides it, otherwise it is a function.
PyPy also provides it as a function.

Note: See TracTickets for help on using tickets.
Back to Top