Incorrect checking or re-conciliation of parent/child fields when inheritance is changed to abstract

[Sawan Vithlani]

Consider the following class hierarchy:

class TestCase(models.Model):
    name = models.CharField(max_length=10)

class TerminalTest(TestCase):
    pass

class BrowserTest(TestCase):
    pass

$ ./manage.py makemigrations testbed
Migrations for 'testbed':
  testbed\migrations\0001_initial.py:
    - Create model TestCase
    - Create model BrowserTest
    - Create model TerminalTest

$ ./manage.py migrate testbed
Operations to perform:
  Apply all migrations: testbed
Running migrations:
Applying testbed.0001_initial... OK

Now we want to make the parent an abstract class:

class TestCase(models.Model):
    name = models.CharField(max_length=10)

    class Meta:
        abstract = True

class TerminalTest(TestCase):
    pass

class BrowserTest(TestCase):
    pass

$ ./manage.py makemigrations testbed
You are trying to add a non-nullable field 'id' to browsertest without a default; we can't do that (the database needs something to populate existing rows).
Please select a fix:
 1) Provide a one-off default now (will be set on all existing rows with a null value for this column)
 2) Quit, and let me add a default in models.py
Select an option: 1
Please enter the default value now, as valid Python
The datetime and django.utils.timezone modules are available, so you can do e.g. timezone.now
Type 'exit' to exit this prompt
>>> 1
You are trying to add a non-nullable field 'name' to browsertest without a default; we can't do that (the database needs something to populate existing rows).
Please select a fix:
 1) Provide a one-off default now (will be set on all existing rows with a null value for this column)
 2) Quit, and let me add a default in models.py
Select an option: 1
Please enter the default value now, as valid Python
The datetime and django.utils.timezone modules are available, so you can do e.g. timezone.now
Type 'exit' to exit this prompt
>>> ' '
You are trying to add a non-nullable field 'id' to terminaltest without a default; we can't do that (the database needs something to populate existing rows).
Please select a fix:
 1) Provide a one-off default now (will be set on all existing rows with a null value for this column)
 2) Quit, and let me add a default in models.py
Select an option: 1
Please enter the default value now, as valid Python
The datetime and django.utils.timezone modules are available, so you can do e.g. timezone.now
Type 'exit' to exit this prompt
>>> 1
You are trying to add a non-nullable field 'name' to terminaltest without a default; we can't do that (the database needs something to populate existing rows).
Please select a fix:
 1) Provide a one-off default now (will be set on all existing rows with a null value for this column)
 2) Quit, and let me add a default in models.py
Select an option: 1
Please enter the default value now, as valid Python
The datetime and django.utils.timezone modules are available, so you can do e.g. timezone.now
Type 'exit' to exit this prompt
>>> ' '
Migrations for 'testbed':
  testbed\migrations\0002_auto_20170315_1711.py:
    - Remove field testcase_ptr from browsertest
    - Remove field testcase_ptr from terminaltest
    - Add field id to browsertest
    - Add field name to browsertest
    - Add field id to terminaltest
    - Add field name to terminaltest
    - Delete model TestCase

At this point, please assume that bug #27940 has been fixed.

In this case, it was fixed by the following temporary hack in 'django/db/backends/sqlite3/schema.py at ​this place:

field_maps = list(mapping.items())
            query = "INSERT INTO %s (%s) SELECT %s FROM %s" % (
                self.quote_name(temp_model._meta.db_table),
                ', '.join(self.quote_name(x) for x, y in field_maps),
                ', '.join(y for x, y in field_maps),
                self.quote_name(model._meta.db_table),
            )

            corrected_query= query.replace(' () SELECT  FROM ', ' SELECT * FROM ')
            print('\n\t\t Changing: {} \n\t\t To: {} \n'.format(query, corrected_query))

            self.execute(corrected_query)

Now to run the migration:

$ ./manage.py migrate testbed

Operations to perform:
  Apply all migrations: testbed
Running migrations:
  Applying testbed.0002_auto_20170315_1711...
                 Changing: INSERT INTO "testbed_browsertest" () SELECT  FROM "testbed_browsertest__old"
                 To: INSERT INTO "testbed_browsertest" SELECT * FROM "testbed_browsertest__old"


                 Changing: INSERT INTO "testbed_terminaltest" () SELECT  FROM "testbed_terminaltest__old"
                 To: INSERT INTO "testbed_terminaltest" SELECT * FROM "testbed_terminaltest__old"

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "C:\my-venv\lib\site-packages\django\core\management\__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "C:\my-venv\lib\site-packages\django\core\management\__init__.py", line 359, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "C:\my-venv\lib\site-packages\django\core\management\base.py", line 294, in run_from_argv
    self.execute(*args, **cmd_options)
  File "C:\my-venv\lib\site-packages\django\core\management\base.py", line 345, in execute
    output = self.handle(*args, **options)
  File "C:\my-venv\lib\site-packages\django\core\management\commands\migrate.py", line 204, in handle
    fake_initial=fake_initial,
  File "C:\my-venv\lib\site-packages\django\db\migrations\executor.py", line 115, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "C:\my-venv\lib\site-packages\django\db\migrations\executor.py", line 145, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "C:\my-venv\lib\site-packages\django\db\migrations\executor.py", line 244, in apply_migration
    state = migration.apply(state, schema_editor)
  File "C:\my-venv\lib\site-packages\django\db\migrations\migration.py", line 119, in apply
    operation.state_forwards(self.app_label, project_state)
  File "C:\my-venv\lib\site-packages\django\db\migrations\operations\fields.py", line 73, in state_forwards
    state.reload_model(app_label, self.model_name_lower)
  File "C:\my-venv\lib\site-packages\django\db\migrations\state.py", line 162, in reload_model
    self.apps.render_multiple(states_to_be_rendered)
  File "C:\my-venv\lib\site-packages\django\db\migrations\state.py", line 277, in render_multiple
    model.render(self)
  File "C:\my-venv\lib\site-packages\django\db\migrations\state.py", line 559, in render
    body,
  File "C:\my-venv\lib\site-packages\django\db\models\base.py", line 226, in __new__
    base.__name__,
django.core.exceptions.FieldError: Local field 'id' in class 'BrowserTest' clashes with field of the same name from base class 'TestCase'.

This seems to happen because some verification code in the django.db.models.base.ModelBase' is unable to distinguish or re-concile between fields in the parent class (TestCase, now abstract) and the child class BrowserTest.

This seems to be very strange the clash is on the 'id' field that isn't even explicitly defined on the models -- it is auto added by the system to serve as a primary key.

[Sawan Vithlani]

Please see bug 27941 for related ticket.

[Tim Graham]

Duplicate of #25247.