Opened 3 weeks ago

Last modified 12 days ago

#36892 assigned Bug

Lazy Tuples in field Choices generate repeated migrations with no changes

Reported by: Matt Armand Owned by: Matt Armand
Component: Migrations Version: 5.0
Severity: Normal Keywords: migrations tuple choices lazy functional deconstruction
Cc: Matt Armand Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

## Background

TLDR, when a model field specifies choices backed by a lazily evaluated tuple, Django>=5.0 serializes them incorrectly into migrations files and repeatedly generates identical migrations on repeated makemigrations runs. Django 4.2 is able to handle these fields correctly.

This bug appears to me to have been introduced with the 5.0 release. At time of writing, this bug exists in the latest release on the 5.2 channel (5.2.10) as well as the 6.0 channel (6.0.1).

This issue was originally found in a Django application utilizing the a list of US States in the django-localflavor library as the choices for a model field. That library uses django.utils.functional::lazy for creating a lazily evaluated tuple, and the bug is reproducible with pure Django code and no external dependencies.

This is a minimalistic example of the lazy tuple used in django-localflavor and that can be used to reproduce the bug:

import operator
from django.db import models
from django.utils.functional import lazy

TUPLE_1 = (("A", "A value"),)
TUPLE_2 = (("B", "B value"),)
LAZY_TUPLE = lazy(
    lambda: tuple(sorted(TUPLE_1 + TUPLE_2, key = operator.itemgetter(1))), tuple
)()

class TestModel(models.model):
    test_field = models.CharField(choices=LAZY_TUPLE)

## Expected Behavior

Prior to Django 5.0 (in 4.2.27 for example), running makemigrations on an app containing this field and model yields migration code containing the following serialization: choices=[('A', 'A value'), ('B', 'B value')], The choices attribute is an array as expected, and repeated makemigrations calls successfully detect no changes to the model.

## Actual Behavior

Beginning in Django 5.0, running makemigrations on an app containing this field and model yields migration code containing the following serialization: choices="(('A', 'A value'), ('B', 'B value'))", The choices attribute is now a string representation of the tuple, and repeated makemigrations calls will re-generate a new and identical AlterField migration for this field ad infinitum.

I've pushed a sample reproduction Django app. You can see in the django_lazy_migration_bug/test_app/migrations/ files generated by Django versions 5.x and 6.x, the erroneous behavior is exhibited, and new migration files are repeatedly generated every time makemigrations is run. Under Django 4.2.27, the field is serialized correctly and repeated migrations don't occur.

## Investigation

I'm still not sure quite what the root cause of this is. Comparing 5.0 to 4.2.27, there doesn't seem to be significant change in django.db.migrations.serializer.py::serializer_factory that would change the MigrationWriter's serialization of this field, nor were there any significant changes to the MigrationWriter itself. The first conditional in serializer_factory (concerning the Promise isinstance check) would evaluate to true in both versions.There were some changes to django.utils.functional.py::lazy, specifically to the handling of resultclasses __wrapper__ functions, so maybe that caused some change in the migration serialization. But I don't see an obvious cause for this yet.

I have attached to this ticket a patch to the Django unit tests adding a case for this, which I've confirmed fails currently. As I have time I can debug further, but I wanted to get the issue reported in case someone else had a quicker fix than I.

Attachments (1)

django-lazy-migrations-bug-unit-test.diff (2.0 KB ) - added by Matt Armand 3 weeks ago.

Download all attachments as: .zip

Change History (9)

by Matt Armand, 3 weeks ago

comment:1 by Ahmed Shammas J, 3 weeks ago

Owner: set to Ahmed Shammas J
Status: newassigned

I am Ahmed Shammas, a 2nd-year BCA student and GSoC 2026 aspirant. I am claiming this ticket to investigate the serialization issue with lazy tuples in migrations. I'll post updates as I make progress.

comment:2 by Ahmed Shammas J, 3 weeks ago

Has patch: set

"I have submitted a PR at https://github.com/django/django/pull/20613 which includes a fix in serializer_factory and a regression test.

comment:3 by Jacob Walls, 3 weeks ago

Has patch: unset
Keywords: deconstruction added
Triage Stage: UnreviewedAccepted

Thanks for the report. The specific location you're seeing this problem inside choices is new since 500e01073adda32d5149624ee9a5cb7aa3d3583f, but the problem applies to all field arguments, as discussed in ticket:29852#comment:6 where a proposed solution is identified:

Promise serialization currently assumes it's only used for string values and it should be adjusted to deal with other types as well. What I suggest doing is inspecting wrapped types of the promise object and only perform a str on the value if the types are (str,). Else the value should be deconstructed as a django.utils.functional.lazy import and a lazy(...)() reconstruction.

in reply to:  3 comment:4 by Matt Armand, 3 weeks ago

Replying to Jacob Walls:

Thanks Jacob, that makes sense. I notice that ticket conversation is pretty old, is the proposed Promise serialization enhancement already planned or in progress, or would that just be the best approach to fix this issue should someone pick it up?

comment:5 by Jacob Walls, 3 weeks ago

I'd be glad to review a contribution implementing that suggestion. I think you can take the liberty to assign the ticket to yourself in that case, as the prior attempt preceded ticket triage.

comment:6 by Ahmed Shammas J, 3 weeks ago

Has patch: set

I have resubmitted a clean, atomic PR with the requested regression test here: https://github.com/django/django/pull/20620.

comment:7 by Matt Armand, 13 days ago

Owner: changed from Ahmed Shammas J to Matt Armand

I went ahead and grabbed this since the previous 2 PRs were missing tests etc. PR with unit tests at https://github.com/django/django/pull/20664. Jacob or whoever let me know if you need anything from me here, I've tried to follow the contributing guidelines closely.

comment:8 by Jacob Walls, 12 days ago

Has patch: unset
Note: See TracTickets for help on using tickets.
Back to Top