#36892 new Bug

Lazy Tuples in field Choices generate repeated migrations with no changes

Reported by: Matt Armand Owned by:
Component: Migrations Version: 5.0
Severity: Normal Keywords: migrations tuple choices lazy functional
Cc: Matt Armand Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

## Background

TLDR, when a model field specifies choices backed by a lazily evaluated tuple, Django>=5.0 serializes them incorrectly into migrations files and repeatedly generates identical migrations on repeated makemigrations runs. Django 4.2 is able to handle these fields correctly.

This bug appears to me to have been introduced with the 5.0 release. At time of writing, this bug exists in the latest release on the 5.2 channel (5.2.10) as well as the 6.0 channel (6.0.1).

This issue was originally found in a Django application utilizing the a list of US States in the django-localflavor library as the choices for a model field. That library uses django.utils.functional::lazy for creating a lazily evaluated tuple, and the bug is reproducible with pure Django code and no external dependencies.

This is a minimalistic example of the lazy tuple used in django-localflavor and that can be used to reproduce the bug:

import operator
from django.db import models
from django.utils.functional import lazy

TUPLE_1 = (("A", "A value"),)
TUPLE_2 = (("B", "B value"),)
LAZY_TUPLE = lazy(
    lambda: tuple(sorted(TUPLE_1 + TUPLE_2, key = operator.itemgetter(1))), tuple
)()

class TestModel(models.model):
    test_field = models.CharField(choices=LAZY_TUPLE)

## Expected Behavior

Prior to Django 5.0 (in 4.2.27 for example), running makemigrations on an app containing this field and model yields migration code containing the following serialization: choices=[('A', 'A value'), ('B', 'B value')], The choices attribute is an array as expected, and repeated makemigrations calls successfully detect no changes to the model.

## Actual Behavior

Beginning in Django 5.0, running makemigrations on an app containing this field and model yields migration code containing the following serialization: choices="(('A', 'A value'), ('B', 'B value'))", The choices attribute is now a string representation of the tuple, and repeated makemigrations calls will re-generate a new and identical AlterField migration for this field ad infinitum.

I've pushed a sample reproduction Django app. You can see in the django_lazy_migration_bug/test_app/migrations/ files generated by Django versions 5.x and 6.x, the erroneous behavior is exhibited, and new migration files are repeatedly generated every time makemigrations is run. Under Django 4.2.27, the field is serialized correctly and repeated migrations don't occur.

## Investigation

I'm still not sure quite what the root cause of this is. Comparing 5.0 to 4.2.27, there doesn't seem to be significant change in django.db.migrations.serializer.py::serializer_factory that would change the MigrationWriter's serialization of this field, nor were there any significant changes to the MigrationWriter itself. The first conditional in serializer_factory (concerning the Promise isinstance check) would evaluate to true in both versions.There were some changes to django.utils.functional.py::lazy, specifically to the handling of resultclasses __wrapper__ functions, so maybe that caused some change in the migration serialization. But I don't see an obvious cause for this yet.

I have attached to this ticket a patch to the Django unit tests adding a case for this, which I've confirmed fails currently. As I have time I can debug further, but I wanted to get the issue reported in case someone else had a quicker fix than I.

Attachments (1)

django-lazy-migrations-bug-unit-test.diff (2.0 KB ) - added by Matt Armand 85 minutes ago.

Download all attachments as: .zip

Change History (1)

Note: See TracTickets for help on using tickets.
Back to Top