Migration executor cannot handle AlterField operation on ForeignObject

[Jacob Walls]

Starting with the four models in the composite primary key docs, altering the model with ForeignObject to become a ForeignKey (while also changing the related model to no longer use a composite primary key), produces a migration that cannot run. Failure is similar to #35992 and #35997.

To reproduce:

• Copy the four models from the ​docs
• makemigrations
• Use new models:

class Product(models.Model):
    name = models.CharField(max_length=100)


class Order(models.Model):
    reference = models.CharField(max_length=20, primary_key=True)


class OrderLineItem(models.Model):
    product = models.ForeignKey(Product, on_delete=models.CASCADE)
    order = models.ForeignKey(Order, on_delete=models.CASCADE, primary_key=True)
    quantity = models.IntegerField()


class Foo(models.Model):
    item_order_id = models.IntegerField()
    item_product_id = models.CharField(max_length=20)
    item = models.ForeignKey(
        OrderLineItem,
        on_delete=models.CASCADE,
    )

• makemigrations
• migrate

Running migrations:
  Applying models.12009_order_product_orderlineitem_foo... OK
  Applying models.12010_remove_orderlineitem_pk_alter_foo_item_and_more...Traceback (most recent call last):
  File "/Users/<user>/prj/arches/manage.py", line 27, in <module>
    execute_from_command_line(sys.argv)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/Users/<user>/django/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
    ~~~~~~~~~~~~~~~^^
  File "/Users/<user>/django/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/Users/<user>/django/django/core/management/base.py", line 416, in run_from_argv
    self.execute(*args, **cmd_options)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/<user>/django/django/core/management/base.py", line 460, in execute
    output = self.handle(*args, **options)
  File "/Users/<user>/django/django/core/management/base.py", line 107, in wrapper
    res = handle_func(*args, **kwargs)
  File "/Users/<user>/django/django/core/management/commands/migrate.py", line 353, in handle
    post_migrate_state = executor.migrate(
        targets,
    ...<3 lines>...
        fake_initial=fake_initial,
    )
  File "/Users/<user>/django/django/db/migrations/executor.py", line 135, in migrate
    state = self._migrate_all_forwards(
        state, plan, full_plan, fake=fake, fake_initial=fake_initial
    )
  File "/Users/<user>/django/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
    state = self.apply_migration(
        state, migration, fake=fake, fake_initial=fake_initial
    )
  File "/Users/<user>/django/django/db/migrations/executor.py", line 255, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/Users/<user>/django/django/db/migrations/migration.py", line 132, in apply
    operation.database_forwards(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self.app_label, schema_editor, old_state, project_state
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/<user>/django/django/db/migrations/operations/fields.py", line 236, in database_forwards
    schema_editor.alter_field(from_model, from_field, to_field)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/<user>/django/django/db/backends/base/schema.py", line 848, in alter_field
    if not self._field_should_be_altered(old_field, new_field):
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/<user>/django/django/db/backends/base/schema.py", line 1690, in _field_should_be_altered
    return self.quote_name(old_field.column) != self.quote_name(
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/Users/<user>/django/django/db/backends/base/schema.py", line 207, in quote_name
    return self.connection.ops.quote_name(name)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/Users/<user>/django/django/db/backends/postgresql/operations.py", line 197, in quote_name
    if name.startswith('"') and name.endswith('"'):
       ^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'startswith'

[Jacob Walls]

Setting back to Normal, realized we have this disclaimer:

Django doesn’t support migrating to, or from, a composite primary key after the table is created.

... although the failure here is less about migrating away from the composite primary key and more about migrating away from the ForeignObject shadowing it. (I feel like the example could likely be adjusted to leave the CPK alone).

Perhaps this is an "optimization" ticket then?

[Jacob Walls]

Just chiming in to add that, as I mentioned, this is reproducible without "migrating from a composite primary key" but rather migrating away from the ForeignObject. E.g., making this single change from

    item = models.ForeignObject(
        OrderLineItem,
        on_delete=models.CASCADE,
        from_fields=("item_order_id", "item_product_id"),
        to_fields=("order_id", "product_id"),
    )

to

    item = models.IntegerField()

reproduces the issue.

[Sarah Boyce]

Thank you! Feels a little similar to #34820