Opened 13 months ago

Closed 13 months ago

Last modified 13 months ago

#22777 closed Bug (fixed)

Foreignkey does (sometimes) not get added to migration of M2M with a through model

Reported by: valberg Owned by: nobody
Component: Migrations Version: master
Severity: Release blocker Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

The following models:

from django.conf import settings
from django.db import models


class Event(models.Model):

    participants = models.ManyToManyField(
        settings.AUTH_USER_MODEL,
        through='testapp.Participant',
        related_name='events'
    )


class Participant(models.Model):

    event = models.ForeignKey(
        'testapp.Event',
        related_name='associations',
    )

    user = models.ForeignKey(
        settings.AUTH_USER_MODEL
    )

Result in the following migration

$ ./manage.py makemigrations           
Migrations for 'testapp':
  0001_initial.py:
    - Create model Event
    - Create model Participant

$ cat testapp/migrations/0001_initial.py
# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations
from django.conf import settings


class Migration(migrations.Migration):

    dependencies = [
        migrations.swappable_dependency(settings.AUTH_USER_MODEL),
    ]

    operations = [
        migrations.CreateModel(
            name='Event',
            fields=[
                ('id', models.AutoField(serialize=False, auto_created=True, primary_key=True, verbose_name='ID')),
                ('participants', models.ManyToManyField(through='testapp.Participant', to=settings.AUTH_USER_MODEL)),
            ],
            options={
            },
            bases=(models.Model,),
        ),
        migrations.CreateModel(
            name='Participant',
            fields=[
                ('id', models.AutoField(serialize=False, auto_created=True, primary_key=True, verbose_name='ID')),
                ('event', models.ForeignKey(to_field='id', to='testapp.Event')),
                ('user', models.ForeignKey(to_field='id', to=settings.AUTH_USER_MODEL)),
            ],
            options={
            },
            bases=(models.Model,),
        ),
    ]

When migrating the following error occurs:

$ ./manage.py migrate       
Operations to perform:
  Synchronize unmigrated apps: sessions, auth, admin, contenttypes
  Apply all migrations: testapp
Synchronizing apps without migrations:
  Creating tables...
    Creating table django_admin_log
    Creating table auth_permission
    Creating table auth_group_permissions
    Creating table auth_group
    Creating table auth_user_groups
    Creating table auth_user_user_permissions
    Creating table auth_user
    Creating table django_content_type
    Creating table django_session
  Installing custom SQL...
  Installing indexes...
Running migrations:
  Applying testapp.0001_initial...Traceback (most recent call last):
  File "/home/valberg/.virtualenvs/django/lib/python3.3/site-packages/django/apps/config.py", line 152, in get_model
    return self.models[model_name.lower()]
KeyError: 'participant'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/valberg/.virtualenvs/django/lib/python3.3/site-packages/django/db/migrations/state.py", line 76, in render
    model = self.apps.get_model(lookup_model[0], lookup_model[1])
  File "/home/valberg/.virtualenvs/django/lib/python3.3/site-packages/django/apps/registry.py", line 190, in get_model
    return self.get_app_config(app_label).get_model(model_name.lower())
  File "/home/valberg/.virtualenvs/django/lib/python3.3/site-packages/django/apps/config.py", line 155, in get_model
    "App '%s' doesn't have a '%s' model." % (self.label, model_name))
LookupError: App 'testapp' doesn't have a 'participant' model.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/valberg/.virtualenvs/django/lib/python3.3/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/home/valberg/.virtualenvs/django/lib/python3.3/site-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/valberg/.virtualenvs/django/lib/python3.3/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/valberg/.virtualenvs/django/lib/python3.3/site-packages/django/core/management/base.py", line 337, in execute
    output = self.handle(*args, **options)
  File "/home/valberg/.virtualenvs/django/lib/python3.3/site-packages/django/core/management/commands/migrate.py", line 146, in handle
    executor.migrate(targets, plan, fake=options.get("fake", False))
  File "/home/valberg/.virtualenvs/django/lib/python3.3/site-packages/django/db/migrations/executor.py", line 62, in migrate
    self.apply_migration(migration, fake=fake)
  File "/home/valberg/.virtualenvs/django/lib/python3.3/site-packages/django/db/migrations/executor.py", line 96, in apply_migration
    migration.apply(project_state, schema_editor)
  File "/home/valberg/.virtualenvs/django/lib/python3.3/site-packages/django/db/migrations/migration.py", line 107, in apply
    operation.database_forwards(self.app_label, schema_editor, project_state, new_state)
  File "/home/valberg/.virtualenvs/django/lib/python3.3/site-packages/django/db/migrations/operations/models.py", line 33, in database_forwards
    apps = to_state.render()
  File "/home/valberg/.virtualenvs/django/lib/python3.3/site-packages/django/db/migrations/state.py", line 86, in render
    model=lookup_model,
ValueError: Lookup failed for model referenced by field testapp.Event.participants: testapp.Participant

The odd thing is that the following models (same structure as above, just different names):

from django.conf import settings
from django.db import models


# Same as Event above
class Foo(models.Model):

    users = models.ManyToManyField(
        settings.AUTH_USER_MODEL,
        through='testapp.Bar',
        related_name='foos'
    )


# Same as Participant above
class Bar(models.Model):

    foo = models.ForeignKey(
        'testapp.Foo',
        related_name='bars',
    )

    user = models.ForeignKey(
        settings.AUTH_USER_MODEL
    )

Result in the following:

$ ./manage.py makemigrations           
Migrations for 'testapp':
  0001_initial.py:
    - Create model Bar
    - Create model Foo
    - Add field foo to bar

$ cat testapp/migrations/0001_initial.py 
# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations
from django.conf import settings


class Migration(migrations.Migration):

    dependencies = [
        migrations.swappable_dependency(settings.AUTH_USER_MODEL),
    ]

    operations = [
        migrations.CreateModel(
            name='Bar',
            fields=[
                ('id', models.AutoField(serialize=False, primary_key=True, auto_created=True, verbose_name='ID')),
                ('user', models.ForeignKey(to_field='id', to=settings.AUTH_USER_MODEL)),
            ],
            options={
            },
            bases=(models.Model,),
        ),
        migrations.CreateModel(
            name='Foo',
            fields=[
                ('id', models.AutoField(serialize=False, primary_key=True, auto_created=True, verbose_name='ID')),
                ('users', models.ManyToManyField(to=settings.AUTH_USER_MODEL, through='testapp.Bar')),
            ],
            options={
            },
            bases=(models.Model,),
        ),
        migrations.AddField(
            model_name='bar',
            name='foo',
            field=models.ForeignKey(to_field='id', to='testapp.Foo'),
            preserve_default=True,
        ),
    ]

$ ./manage.py migrate
Operations to perform:
  Synchronize unmigrated apps: admin, auth, sessions, contenttypes
  Apply all migrations: testapp
Synchronizing apps without migrations:
  Creating tables...
    Creating table django_admin_log
    Creating table auth_permission
    Creating table auth_group_permissions
    Creating table auth_group
    Creating table auth_user_groups
    Creating table auth_user_user_permissions
    Creating table auth_user
    Creating table django_content_type
    Creating table django_session
  Installing custom SQL...
  Installing indexes...
Running migrations:
  Applying testapp.0001_initial... OK

So somehow the naming of the models has a say in how the migrations are made. In the failing example there is no creation of the FK from Participant to Event as there is from Bar to Foo.

Change History (4)

comment:1 Changed 13 months ago by bmispelon

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Severity changed from Normal to Release blocker
  • Triage Stage changed from Unreviewed to Accepted

Hi,

I can reproduce your issue with this simplified models (no reference to settings.AUTH_USER_MODEL):

from django.db import models

class Foo(models.Model):
    pass


class Bar(models.Model):
    foos = models.ManyToManyField('Foo', through='FooBar')


class FooBar(models.Model):
    foo = models.ForeignKey('Foo')
    bar = models.ForeignKey('Bar')

As you mentionned, renaming FooBar to AFooBar (so that it appears before when sorting alphabetically) makes everything work.

I'll upgrade the ticket to a release blocker.

Thanks.

comment:2 Changed 13 months ago by andrewgodwin

Looks like there's not enough dependency stuff set in the autodetector to force the operation ordering. Hopefully an easy-ish fix.

comment:3 Changed 13 months ago by Andrew Godwin <andrew@…>

  • Resolution set to fixed
  • Status changed from new to closed

In 3f91238adf819b1d86f800d6b535b3c04447dac2:

Fixed #22777: Add dependency on through for autodetected M2M adds

comment:4 Changed 13 months ago by Andrew Godwin <andrew@…>

In 55fa4c2d3423b051fce5663b3efc13e458fbb777:

[1.7.x] Fixed #22777: Add dependency on through for autodetected M2M adds

Note: See TracTickets for help on using tickets.
Back to Top