Redundant migrations generated with Coalesce Index in GinIndex with SearchVector

[Ian Bicket]

Title: Redundant Migration Generation for Coalesce Index with TextField Output in Django 5.1 (PostgreSQL)

Django Version: 5.1

Database Backend: PostgreSQL (version 15)

Python Version: 3.12

Operating System: Observed on MacOS Sequoia, but likely to generalize

Description:
When running makemigrations on a Django 5.1 project using a PostgreSQL backend, the migration autodetector redundantly generates a migration that removes and then recreates a GinIndex with a SearchVector over a Coalesced field where the output_field parameter is set to TextField(). Furthermore, makemigrations can be run an arbitrary number of times, each execution re-creating what is essentially the same migration, removing and recreating the index.

Steps to Reproduce:

Create a Django model with an index using Coalesce and explicitly set output_field=TextField(), such as in the example below:

from django.contrib.postgres.indexes import GinIndex
from django.contrib.postgres.search import SearchVector
from django.db.models import Value
from django.db.models.functions import Coalesce

# ... model definition, etc.

class SummaryItem(models.Model):
    class Meta:
        indexes = [
                GinIndex(
                    SearchVector(
                        Coalesce(
                            'item_text_by_user',
                            'item_text',
                            Value(''),
                            output_field=TextField()
                    ),
                    'source',
                    config='english',
                ),
                name='summary_item_search_index',
            )
        ]

    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)
    deleted_at = models.DateTimeField(null=True)
    item_type = models.CharField(max_length=16, choices=ItemType)
    item_text = models.TextField()
    item_text_by_user = models.TextField(null=True)

Run makemigrations and apply the migration.

Run makemigrations again without making any changes.

Observe that a new migration is generated that removes and recreates the same Coalesce index.

Expected Behavior:
Once the initial migration has been applied, subsequent runs of makemigrations should not generate redundant migrations unless there is an actual schema change.

Actual Behavior:
Each execution of makemigrations generates an unnecessary migration removing and recreating the Coalesce index.

Potential Cause:
The migration autodetector may be treating the TextField instances in the existing and new application state as distinct, even though they are equivalent. This causes Django to incorrectly assume a change has occurred and generate a new migration. It was observed while breaking down the deconstructed indices from the autodetector's from_state and to_state (see here: ​https://github.com/django/django/blob/0bf412111be686b6b23e00863f5d449d63557dbf/django/db/migrations/autodetector.py#L1347) that the deconstructed new and old indexes were identified as different because the creation_counter attributes of the TextField() passed to output_field in the indexes (which in turn occurred in the ' in the two respective states were different, which is part of how the __eq__ operator is defined for objects inheriting from Field, and therefore the index in the old and new states were identified as separate, even though there was no change in the definition of the index.

Workaround:
As a temporary workaround, removing output_field=TextField() and the Value() (which defaults to a CharField rendering the output_field necessary) parameter from Coalesce will prevent redundant migrations if Django can infer the correct type automatically. However, this might not be suitable in all cases.

[Simon Charette]

Duplicate of #36173 (fixed by df2c4952df6d93c575fb8a3c853dc9d4c2449f36) which will be part of Django 5.2b1 meant to be released tomorrow. Just like Concat the Coalesce's __init__ method was using *args and **kwargs capture which broke BaseExpression.identity.