removal of concrete Model from bases doesn't remove it from ModelState bases

[Sergey Fedoseev]

Steps to reproduce:

• add test app with models.py:

  class Thing(models.Model):
      pass
  
  
  class SuperThing(Thing):
      pass
  

• do

manage.py makemigrations test

• change models.py (remove Thing):

  class SuperThing(models.Model):
      thing_ptr = models.AutoField(primary_key=True)
  

• do

manage.py makemigrations test

• do it one more time

manage.py makemigrations test

last command results in

django.db.migrations.state.InvalidBasesError: Cannot resolve bases for [<ModelState: 'test.SuperThing'>]
This can happen if you are inheriting models from an app with migrations (e.g. contrib.auth)
 in an app with no migrations; see https://docs.djangoproject.com/en/1.7/topics/migrations/#dependencies for more

[Baptiste Mispelon]

Hi,

I can't reproduce the issue you're describing.

On master, the second makemigrations doesn't detect any changes.
On stable/1.7.x, it does create a migration but running it a third time works (and detects no change as expected).

As a side note, it's not possible to create an app called test because you get this error when doing manage.py startapp test:

CommandError: 'test' conflicts with the name of an existing Python module and cannot be used as an app name. Please try another name.

Can you provide us with some details on how you're trigerring the issue?

Thanks.

[Sergey Fedoseev]

Replying to bmispelon:

Hi,

Probably this is not obvious, but removal of Thing model from models.py, not only from SuperThing bases was implied.

As a side note, it's not possible to create an app called test because you get this error when doing manage.py startapp test

I have never used this for creating apps =)

[Baptiste Mispelon]

Thanks, that's the bit I was missing.

[Markus Holtermann]

Are you working on a patch sir-sigurd? If not, I'd write one.

[Sergey Fedoseev]

Replying to Markush2010:

Are you working on a patch sir-sigurd? If not, I'd write one.

Yes, I'm working on it.

[Yuval Adam]

There's an easier way to reproduce what seems to be the exact same error:

• Create a BaseModel and a SubModel that inherits from it
• Run makemigrations and migrate to add both models to database
• Delete both models together
• Run makemigrations and migrate to delete both models from the database

The same error should now appear.

[ris]

I invite you all to look at my marked-as-duplicate bug #23818. To me it seems there is simply no way for django's migration operations to express a change of bases. parent_links are created & removed, but the built-up in-memory representation of the models never understands the new parentage, meaning certain operations (e.g. in a RunPython operation) will simply not work right when referring to inherited fields. This goes beyond makemigrations simply not detecting a change.

[Eric Mills]

This bug has burned me pretty bad. I was working through steps to switch from concrete inheritance to abstract inheritance. Is someone going to revisit this?

[Tim Graham]

#23521 may be related or a duplicate.

[Federico Jaramillo Martínez]

Having the same issue.
I had to manually alter my initial migration and remove the offending base class for my app to work.

[rm_]

I had the need as described in #23818 but instead of squashing migrations i've created a migrations operator to update the bases on migrations models state. This works fine for me on django 1.8. I still haven't tested if django master behaves differently. Do you think this is a valuable addition to django?

class AlterBaseOperation(Operation):
    reduce_to_sql = False
    reversible = True

    def __init__(self, model_name, bases, prev_bases):
        self.model_name = model_name
        self.bases = bases
        self.prev_bases = prev_bases

    def state_forwards(self, app_label, state):
        state.models[app_label, self.model_name].bases = self.bases
        state.reload_model(app_label, self.model_name)

    def state_backwards(self, app_label, state):
        state.models[app_label, self.model_name].bases = self.prev_bases
        state.reload_model(app_label, self.model_name)

    def database_forwards(self, app_label, schema_editor, from_state, to_state):
        pass

    def database_backwards(self, app_label, schema_editor, from_state, to_state):
        pass

    def describe(self):
        return "Update %s bases to %s" % (self.model_name, self.bases)

[Ian Foote]

​PR

[Mariusz Felisiak]

Another use case of migrations.AlterModelBases() is described in #30513.

[Karolis Ryselis]

There is one more case that needs to be addressed related to bases in migrations.

Step 1:
Consider this model:

class Good(GoodServiceBase):
    title = models.CharField(max_length=16)

this creates migration with the following operation:

migrations.CreateModel(
    name='Good',
    fields=[
        ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
        ('title', models.CharField(max_length=16)),
    ],
),

Step 2:
Then we add a new model and change existing model to inherit from the new model.

class GoodServiceBase(models.Model):
    active = models.BooleanField(default=True)

class Good(GoodServiceBase):
    title = models.CharField(max_length=16)

This generates migration like this:

migrations.CreateModel(
    name='GoodServiceBase',
    fields=[
        ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
        ('active', models.BooleanField(default=True)),
    ],
),
migrations.RemoveField(
    model_name='good',
    name='id',
),
migrations.AddField(
    model_name='good',
    name='goodservicebase_ptr',
    field=models.OneToOneField(auto_created=True, default=1, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='goods.GoodServiceBase'),
    preserve_default=False,
),

If we have no data in database, this will pass.

Step 3 with crash:
Try to access field active in datamigration and migration crashes. Operation in migration to reproduce this:

def migrate_my_data(apps, schema_editor):
    for good in apps.get_model("goods", "Good").objects.all():
        print(good.active)

migrations.RunPython(migrate_my_data, reverse_code=lambda x, y: None)

This crashes with the error:

<...>
File "<project home>/goods/migrations/0003_auto_20191025_1157.py", line 8, in migrate_my_data
  print(good.active)
AttributeError: 'Good' object has no attribute 'active'

As far as I understand this happens because bases is not altered in migration and fake models do not inherit fields from their true parent models. Thus, currently if we change inheritance chain of existing models, we lose the ability to use those models in data migrations.

Tested in version 2.2.6.

[Mariusz Felisiak]

Another two duplicates: #31329 and #26488.

[Mariusz Felisiak]

Another use case is described in #31343.

[Ian Foote]

Deassigning because I don't think I'm likely to get to this again in the near future and I'd be happy for someone else to take my work and finish the job.

[Vadim Fabrichnov]

​PR