makemigrations removes fields in wrong order when GeneratedFields are involved

[Colton Saska]

Consider a model like PayItem with fields quantity, rate and total.

When total is a GeneratedField dependent on quantity, if you remove quantity and total and generate migrations you get a migration like

    operations = [
        migrations.RemoveField(
            model_name="payitem",
            name="total",
        ),
        migrations.RemoveField(
            model_name="payitem",
            name="quantity",
        ),
    ]

which will throw django.db.utils.OperationalError: (3108, "Column 'amount' has a generated column dependency.").

It'd be nice if makemigrations listed operations in the correct order.

[Colton Saska]

Replying to csaska:

Consider a model like PayItem with fields quantity, rate and total.

When total is a GeneratedField dependent on quantity, if you remove quantity and total and generate migrations you get a migration like

    operations = [
        migrations.RemoveField(
            model_name="payitem",
            name="total",
        ),
        migrations.RemoveField(
            model_name="payitem",
            name="quantity",
        ),
    ]

which will throw django.db.utils.OperationalError: (3108, "Column 'amount' has a generated column dependency.").

It'd be nice if makemigrations listed operations in the correct order.

In my example, I actually had fields amount, rate and total. Perhaps it using alphabetical order and thus dropping amount operation comes before dropping total operation?

[Simon Charette]

I haven't reproduced myself but given I can't find any logic in AutoDetector._generate_removed_field to ensure that GeneratedField removals depend on removals of fields they reference this seems legitimate. I suspect field addition through makemigrations suffers from the same problem and that the optimizer can also result in ordering issues because FieldOperation.references_field doesn't consider generated field references.

[Sarah Boyce <42296566+sarahboyce@…>]

In 1a7fc0f:

Fixed #36438 -- Made MigrationAutodetector remove generated fields before their base fields.

Thanks to Colton Saska for the report and to Simon Charette for the review.

[Sarah Boyce <42296566+sarahboyce@…>]

In 45ba7683:

Refs #36438 -- Made FieldOperation.references_field() detect references in GeneratedField.expression.

Thanks to Simon Charette for the suggestion and review.

[Jacob Walls <jacobtylerwalls@…>]

In 336e713:

Refs #36438 -- Simplified retrieval of GeneratedField base fields in migrations autodetector.

This was creating an unusable Q object solely to call
referenced_base_fields on it.