makemigrations creates broken initial migration for models with multiple inheritance

[Kirill Gagarski]

Hello.
I am trying to create models that use multiple inheritance and then use django migrations mechanism to migrate. But I have some problems with it.

I have created simplest example to reproduce.

Steps to reproduce

1. Create a migrationstest django app with following models.py:

from django.db import models


class A(models.Model):
    a_id = models.AutoField(primary_key=True)


class B(models.Model):
    b_id = models.AutoField(primary_key=True)


class C(A, B):
    pass


class D(A, B):
    pass


class E(A, B):
    pass

2. run './manage.py makemigrations migrationstest'
3. run './manage.py migrate migrationstest'

Expected results

Fine migration

Actual results

Exception is thrown while running 'migrate':

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/gagarski/polyana-web/environment/lib/python3.4/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/home/gagarski/polyana-web/environment/lib/python3.4/site-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/gagarski/polyana-web/environment/lib/python3.4/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/gagarski/polyana-web/environment/lib/python3.4/site-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/home/gagarski/polyana-web/environment/lib/python3.4/site-packages/django/core/management/commands/migrate.py", line 160, in handle
    executor.migrate(targets, plan, fake=options.get("fake", False))
  File "/home/gagarski/polyana-web/environment/lib/python3.4/site-packages/django/db/migrations/executor.py", line 63, in migrate
    self.apply_migration(migration, fake=fake)
  File "/home/gagarski/polyana-web/environment/lib/python3.4/site-packages/django/db/migrations/executor.py", line 97, in apply_migration
    migration.apply(project_state, schema_editor)
  File "/home/gagarski/polyana-web/environment/lib/python3.4/site-packages/django/db/migrations/migration.py", line 107, in apply
    operation.database_forwards(self.app_label, schema_editor, project_state, new_state)
  File "/home/gagarski/polyana-web/environment/lib/python3.4/site-packages/django/db/migrations/operations/fields.py", line 37, in database_forwards
    field,
  File "/home/gagarski/polyana-web/environment/lib/python3.4/site-packages/django/db/backends/schema.py", line 390, in add_field
    self.execute(sql, params)
  File "/home/gagarski/polyana-web/environment/lib/python3.4/site-packages/django/db/backends/schema.py", line 99, in execute
    cursor.execute(sql, params)
  File "/home/gagarski/polyana-web/environment/lib/python3.4/site-packages/django/db/backends/utils.py", line 81, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/home/gagarski/polyana-web/environment/lib/python3.4/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/home/gagarski/polyana-web/environment/lib/python3.4/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/gagarski/polyana-web/environment/lib/python3.4/site-packages/django/utils/six.py", line 549, in reraise
    raise value.with_traceback(tb)
  File "/home/gagarski/polyana-web/environment/lib/python3.4/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column "b_ptr_id" of relation "migrationstest_d" already exists

More details

The generated initial migration:

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations


class Migration(migrations.Migration):

    dependencies = [
    ]

    operations = [
        migrations.CreateModel(
            name='A',
            fields=[
                ('a_id', models.AutoField(primary_key=True, serialize=False)),
            ],
            options={
            },
            bases=(models.Model,),
        ),
        migrations.CreateModel(
            name='B',
            fields=[
                ('b_id', models.AutoField(primary_key=True, serialize=False)),
            ],
            options={
            },
            bases=(models.Model,),
        ),
        migrations.CreateModel(
            name='C',
            fields=[
                ('a_ptr', models.OneToOneField(primary_key=True, auto_created=True, serialize=False, to='migrationstest.A', parent_link=True)),
            ],
            options={
            },
            bases=('migrationstest.a', 'migrationstest.b'),
        ),
        migrations.CreateModel(
            name='D',
            fields=[
                ('a_ptr', models.OneToOneField(primary_key=True, auto_created=True, serialize=False, to='migrationstest.A', parent_link=True)),
            ],
            options={
            },
            bases=('migrationstest.a', 'migrationstest.b'),
        ),
        migrations.CreateModel(
            name='E',
            fields=[
                ('a_ptr', models.OneToOneField(primary_key=True, auto_created=True, serialize=False, to='migrationstest.A', parent_link=True)),
                ('b_ptr', models.OneToOneField(parent_link=True, auto_created=True, to='migrationstest.B')),
            ],
            options={
            },
            bases=('migrationstest.a', 'migrationstest.b'),
        ),
        migrations.AddField(
            model_name='d',
            name='b_ptr',
            field=models.OneToOneField(parent_link=True, auto_created=True, to='migrationstest.B'),
            preserve_default=True,
        ),
        migrations.AddField(
            model_name='c',
            name='b_ptr',
            field=models.OneToOneField(parent_link=True, auto_created=True, to='migrationstest.B'),
            preserve_default=True,
        ),
    ]

As you can see b_ptr field is added in CreateModel only for the last model (E).
For D and C it is added via AddField. But it seems that b_ptr is already added to them even without a field specified in CreateModel's fields argument (with the help of bases argument, I think).

Output of sqlmigrate

BEGIN;
CREATE TABLE "migrationstest_a" ("a_id" serial NOT NULL PRIMARY KEY);
CREATE TABLE "migrationstest_b" ("b_id" serial NOT NULL PRIMARY KEY);
CREATE TABLE "migrationstest_c" ("b_ptr_id" integer NOT NULL UNIQUE, "a_ptr_id" integer NOT NULL PRIMARY KEY);
CREATE TABLE "migrationstest_d" ("b_ptr_id" integer NOT NULL UNIQUE, "a_ptr_id" integer NOT NULL PRIMARY KEY);
CREATE TABLE "migrationstest_e" ("b_ptr_id" integer NOT NULL UNIQUE, "a_ptr_id" integer NOT NULL PRIMARY KEY);
ALTER TABLE "migrationstest_d" ADD COLUMN "b_ptr_id" integer NOT NULL UNIQUE;
ALTER TABLE "migrationstest_d" ALTER COLUMN "b_ptr_id" DROP DEFAULT;
ALTER TABLE "migrationstest_c" ADD COLUMN "b_ptr_id" integer NOT NULL UNIQUE;
ALTER TABLE "migrationstest_c" ALTER COLUMN "b_ptr_id" DROP DEFAULT;
ALTER TABLE "migrationstest_c" ADD CONSTRAINT "migrationste_b_ptr_id_5529c1e12e7a3aaa_fk_migrationstest_b_b_id" FOREIGN KEY ("b_ptr_id") REFERENCES "migrationstest_b" ("b_id") DEFERRABLE INITIALLY DEFERRED;
ALTER TABLE "migrationstest_c" ADD CONSTRAINT "migrationste_a_ptr_id_12130065dfb7292a_fk_migrationstest_a_a_id" FOREIGN KEY ("a_ptr_id") REFERENCES "migrationstest_a" ("a_id") DEFERRABLE INITIALLY DEFERRED;
ALTER TABLE "migrationstest_d" ADD CONSTRAINT "migrationste_b_ptr_id_4eee8ae4e0885aaa_fk_migrationstest_b_b_id" FOREIGN KEY ("b_ptr_id") REFERENCES "migrationstest_b" ("b_id") DEFERRABLE INITIALLY DEFERRED;
ALTER TABLE "migrationstest_d" ADD CONSTRAINT "migrationste_a_ptr_id_7f0ea1e45cc6092a_fk_migrationstest_a_a_id" FOREIGN KEY ("a_ptr_id") REFERENCES "migrationstest_a" ("a_id") DEFERRABLE INITIALLY DEFERRED;
ALTER TABLE "migrationstest_e" ADD CONSTRAINT "migrationste_b_ptr_id_7eec341934e15699_fk_migrationstest_b_b_id" FOREIGN KEY ("b_ptr_id") REFERENCES "migrationstest_b" ("b_id") DEFERRABLE INITIALLY DEFERRED;
ALTER TABLE "migrationstest_e" ADD CONSTRAINT "migrationste_a_ptr_id_2c800772a4d234e7_fk_migrationstest_a_a_id" FOREIGN KEY ("a_ptr_id") REFERENCES "migrationstest_a" ("a_id") DEFERRABLE INITIALLY DEFERRED;
ALTER TABLE "migrationstest_d" ADD CONSTRAINT "migrationste_b_ptr_id_4eee8ae4e0885aaa_fk_migrationstest_b_b_id" FOREIGN KEY ("b_ptr_id") REFERENCES "migrationstest_b" ("b_id") DEFERRABLE INITIALLY DEFERRED;
ALTER TABLE "migrationstest_c" ADD CONSTRAINT "migrationste_b_ptr_id_5529c1e12e7a3aaa_fk_migrationstest_b_b_id" FOREIGN KEY ("b_ptr_id") REFERENCES "migrationstest_b" ("b_id") DEFERRABLE INITIALLY DEFERRED;

COMMIT;

Questions

Is there a good reasons to do like that? Is there a workaround that can help me to use automatic migration generation without manually editing them (at least initial migration)?

[Kirill Gagarski]

Changed "IntegerField" to "AutoField"

[Tim Graham]

I can reproduce the error with PostgreSQL.

[Markus Holtermann]

Hey gagarski, thanks for the report. Unfortunately I don't see a way to use the automatically created migration files w/o modifications.

Workaround:

1. Move the field definition from the AddField operation into the respective CreateModel definition (the way it looks for the model E)
2. Remove the AddField operation

[Torsten Bronger]

I added primary_key=True to a field my_field of a derived model class and the migration automatically generated didn't work, due to this issue.

If I apply the workaround from comment:6, I effectively move the primary_key=True from the ..._ptr field to my_field. The migration does work then.

However, I have to move the serialize=False parameter, too. But I want my_field to be serialized with dumpdata (I suspect this is meant by this parameter). If I don't move serialize=False, ./manage.py makemigration myapp claims that something has to be changed.

[Markus Holtermann]

Here's a pull-request to fix the issue: ​https://github.com/django/django/pull/3715

[Tim Graham <timograham@…>]

In 44927ba817a4ecf9834d429ff6c86bc5ac961305:

Fixed #23956 -- Fixed migration creation for multiple table inheritance

[Tim Graham <timograham@…>]

In f446acf8bbd6360fdb5a9d34186c5fadf98282b7:

[1.7.x] Fixed #23956 -- Fixed migration creation for multiple table inheritance

Backport of 44927ba817a4ecf9834d429ff6c86bc5ac961305 from master

[Torsten Bronger]

I'm still confused why makemigrations generates serialize=False for my custom primary key.