Opened 86 minutes ago
#36892 new Bug
Lazy Tuples in field Choices generate repeated migrations with no changes
| Reported by: | Matt Armand | Owned by: | |
|---|---|---|---|
| Component: | Migrations | Version: | 5.0 |
| Severity: | Normal | Keywords: | migrations tuple choices lazy functional |
| Cc: | Matt Armand | Triage Stage: | Unreviewed |
| Has patch: | no | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description
## Background
TLDR, when a model field specifies choices backed by a lazily evaluated tuple, Django>=5.0 serializes them incorrectly into migrations files and repeatedly generates identical migrations on repeated makemigrations runs. Django 4.2 is able to handle these fields correctly.
This bug appears to me to have been introduced with the 5.0 release. At time of writing, this bug exists in the latest release on the 5.2 channel (5.2.10) as well as the 6.0 channel (6.0.1).
This issue was originally found in a Django application utilizing the a list of US States in the django-localflavor library as the choices for a model field. That library uses django.utils.functional::lazy for creating a lazily evaluated tuple, and the bug is reproducible with pure Django code and no external dependencies.
This is a minimalistic example of the lazy tuple used in django-localflavor and that can be used to reproduce the bug:
import operator
from django.db import models
from django.utils.functional import lazy
TUPLE_1 = (("A", "A value"),)
TUPLE_2 = (("B", "B value"),)
LAZY_TUPLE = lazy(
lambda: tuple(sorted(TUPLE_1 + TUPLE_2, key = operator.itemgetter(1))), tuple
)()
class TestModel(models.model):
test_field = models.CharField(choices=LAZY_TUPLE)
## Expected Behavior
Prior to Django 5.0 (in 4.2.27 for example), running makemigrations on an app containing this field and model yields migration code containing the following serialization: choices=[('A', 'A value'), ('B', 'B value')], The choices attribute is an array as expected, and repeated makemigrations calls successfully detect no changes to the model.
## Actual Behavior
Beginning in Django 5.0, running makemigrations on an app containing this field and model yields migration code containing the following serialization: choices="(('A', 'A value'), ('B', 'B value'))", The choices attribute is now a string representation of the tuple, and repeated makemigrations calls will re-generate a new and identical AlterField migration for this field ad infinitum.
I've pushed a sample reproduction Django app. You can see in the django_lazy_migration_bug/test_app/migrations/ files generated by Django versions 5.x and 6.x, the erroneous behavior is exhibited, and new migration files are repeatedly generated every time makemigrations is run. Under Django 4.2.27, the field is serialized correctly and repeated migrations don't occur.
## Investigation
I'm still not sure quite what the root cause of this is. Comparing 5.0 to 4.2.27, there doesn't seem to be significant change in django.db.migrations.serializer.py::serializer_factory that would change the MigrationWriter's serialization of this field, nor were there any significant changes to the MigrationWriter itself. The first conditional in serializer_factory (concerning the Promise isinstance check) would evaluate to true in both versions.There were some changes to django.utils.functional.py::lazy, specifically to the handling of resultclasses __wrapper__ functions, so maybe that caused some change in the migration serialization. But I don't see an obvious cause for this yet.
I have attached to this ticket a patch to the Django unit tests adding a case for this, which I've confirmed fails currently. As I have time I can debug further, but I wanted to get the issue reported in case someone else had a quicker fix than I.