Opened 3 weeks ago
Last modified 12 days ago
#36892 assigned Bug
Lazy Tuples in field Choices generate repeated migrations with no changes
| Reported by: | Matt Armand | Owned by: | Matt Armand |
|---|---|---|---|
| Component: | Migrations | Version: | 5.0 |
| Severity: | Normal | Keywords: | migrations tuple choices lazy functional deconstruction |
| Cc: | Matt Armand | Triage Stage: | Accepted |
| Has patch: | no | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description
## Background
TLDR, when a model field specifies choices backed by a lazily evaluated tuple, Django>=5.0 serializes them incorrectly into migrations files and repeatedly generates identical migrations on repeated makemigrations runs. Django 4.2 is able to handle these fields correctly.
This bug appears to me to have been introduced with the 5.0 release. At time of writing, this bug exists in the latest release on the 5.2 channel (5.2.10) as well as the 6.0 channel (6.0.1).
This issue was originally found in a Django application utilizing the a list of US States in the django-localflavor library as the choices for a model field. That library uses django.utils.functional::lazy for creating a lazily evaluated tuple, and the bug is reproducible with pure Django code and no external dependencies.
This is a minimalistic example of the lazy tuple used in django-localflavor and that can be used to reproduce the bug:
import operator
from django.db import models
from django.utils.functional import lazy
TUPLE_1 = (("A", "A value"),)
TUPLE_2 = (("B", "B value"),)
LAZY_TUPLE = lazy(
lambda: tuple(sorted(TUPLE_1 + TUPLE_2, key = operator.itemgetter(1))), tuple
)()
class TestModel(models.model):
test_field = models.CharField(choices=LAZY_TUPLE)
## Expected Behavior
Prior to Django 5.0 (in 4.2.27 for example), running makemigrations on an app containing this field and model yields migration code containing the following serialization: choices=[('A', 'A value'), ('B', 'B value')], The choices attribute is an array as expected, and repeated makemigrations calls successfully detect no changes to the model.
## Actual Behavior
Beginning in Django 5.0, running makemigrations on an app containing this field and model yields migration code containing the following serialization: choices="(('A', 'A value'), ('B', 'B value'))", The choices attribute is now a string representation of the tuple, and repeated makemigrations calls will re-generate a new and identical AlterField migration for this field ad infinitum.
I've pushed a sample reproduction Django app. You can see in the django_lazy_migration_bug/test_app/migrations/ files generated by Django versions 5.x and 6.x, the erroneous behavior is exhibited, and new migration files are repeatedly generated every time makemigrations is run. Under Django 4.2.27, the field is serialized correctly and repeated migrations don't occur.
## Investigation
I'm still not sure quite what the root cause of this is. Comparing 5.0 to 4.2.27, there doesn't seem to be significant change in django.db.migrations.serializer.py::serializer_factory that would change the MigrationWriter's serialization of this field, nor were there any significant changes to the MigrationWriter itself. The first conditional in serializer_factory (concerning the Promise isinstance check) would evaluate to true in both versions.There were some changes to django.utils.functional.py::lazy, specifically to the handling of resultclasses __wrapper__ functions, so maybe that caused some change in the migration serialization. But I don't see an obvious cause for this yet.
I have attached to this ticket a patch to the Django unit tests adding a case for this, which I've confirmed fails currently. As I have time I can debug further, but I wanted to get the issue reported in case someone else had a quicker fix than I.
Attachments (1)
Change History (9)
by , 3 weeks ago
| Attachment: | django-lazy-migrations-bug-unit-test.diff added |
|---|
comment:1 by , 3 weeks ago
| Owner: | set to |
|---|---|
| Status: | new → assigned |
comment:2 by , 3 weeks ago
| Has patch: | set |
|---|
"I have submitted a PR at https://github.com/django/django/pull/20613 which includes a fix in serializer_factory and a regression test.
follow-up: 4 comment:3 by , 3 weeks ago
| Has patch: | unset |
|---|---|
| Keywords: | deconstruction added |
| Triage Stage: | Unreviewed → Accepted |
Thanks for the report. The specific location you're seeing this problem inside choices is new since 500e01073adda32d5149624ee9a5cb7aa3d3583f, but the problem applies to all field arguments, as discussed in ticket:29852#comment:6 where a proposed solution is identified:
Promise serialization currently assumes it's only used for string values and it should be adjusted to deal with other types as well. What I suggest doing is inspecting wrapped types of the promise object and only perform a str on the value if the types are (str,). Else the value should be deconstructed as a django.utils.functional.lazy import and a lazy(...)() reconstruction.
comment:4 by , 3 weeks ago
Replying to Jacob Walls:
Thanks Jacob, that makes sense. I notice that ticket conversation is pretty old, is the proposed Promise serialization enhancement already planned or in progress, or would that just be the best approach to fix this issue should someone pick it up?
comment:5 by , 3 weeks ago
I'd be glad to review a contribution implementing that suggestion. I think you can take the liberty to assign the ticket to yourself in that case, as the prior attempt preceded ticket triage.
comment:6 by , 3 weeks ago
| Has patch: | set |
|---|
I have resubmitted a clean, atomic PR with the requested regression test here: https://github.com/django/django/pull/20620.
comment:7 by , 13 days ago
| Owner: | changed from to |
|---|
I went ahead and grabbed this since the previous 2 PRs were missing tests etc. PR with unit tests at https://github.com/django/django/pull/20664. Jacob or whoever let me know if you need anything from me here, I've tried to follow the contributing guidelines closely.
comment:8 by , 12 days ago
| Has patch: | unset |
|---|
I am Ahmed Shammas, a 2nd-year BCA student and GSoC 2026 aspirant. I am claiming this ticket to investigate the serialization issue with lazy tuples in migrations. I'll post updates as I make progress.