Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#24725 closed Bug (fixed)

"ProgrammingError: relation does not exist" when renaming many-to-many target model

Reported by: Danilo Bargen Owned by: Markus Holtermann
Component: Migrations Version: 1.8
Severity: Release blocker Keywords: migrations
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

To demonstrate this bug, I created a Github repository: https://github.com/dbrgn/django18-migration-bug You can see each step in the commit log.

When I run the migrations for that project in a clean Postgres database, I get an exception. You can run "reproduce.sh" to reproduce (d'oh!).

$ dropdb dj18
$ createdb dj18
$ ./manage.py migrate
Operations to perform:
  Synchronize unmigrated apps: staticfiles, messages
  Apply all migrations: admin, debug, contenttypes, auth, sessions
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying debug.0001_initial... OK
  Applying debug.0002_auto_20150429_1317...Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/core/management/__init__.py", line 330, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/core/management/base.py", line 390, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/core/management/base.py", line 441, in execute
    output = self.handle(*args, **options)
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 221, in handle
    executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/db/migrations/executor.py", line 110, in migrate
    self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/db/migrations/executor.py", line 147, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/db/migrations/migration.py", line 115, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/db/migrations/operations/models.py", line 217, in database_forwards
    to_field,
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 469, in alter_field
    return self._alter_many_to_many(model, old_field, new_field, strict)
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 780, in _alter_many_to_many
    new_field.rel.through._meta.get_field(new_field.m2m_reverse_field_name()),
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 484, in alter_field
    old_db_params, new_db_params, strict)
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 715, in _alter_field
    self.execute(self._create_fk_sql(model, new_field, "_fk_%(to_table)s_%(to_column)s"))
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 107, in execute
    cursor.execute(sql, params)
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/danilo/.virtualenvs/django18-migration-bug/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "debug_spam" does not exist

This only happens in Django 1.8, with 1.7.7 the migrations run fine.

Attachments (2)

traceback-sqlite.txt (2.8 KB) - added by Marten Kenbeek 3 years ago.
On sqlite the initial migration works, but subsequent calls to migrate or makemigrations produce this traceback.
24725-test.diff (1.7 KB) - added by Tim Graham 3 years ago.

Download all attachments as: .zip

Change History (13)

comment:1 Changed 3 years ago by Danilo Bargen

Note that both migrations were auto-generated:

$ ./manage.py makemigrations debug
Did you rename the debug.Spam model to Bacon? [y/N] y
Migrations for 'debug':
  0002_auto_20150429_1331.py:
    - Rename model Spam to Bacon

comment:2 Changed 3 years ago by Marten Kenbeek

Triage Stage: UnreviewedAccepted

Changed 3 years ago by Marten Kenbeek

Attachment: traceback-sqlite.txt added

On sqlite the initial migration works, but subsequent calls to migrate or makemigrations produce this traceback.

comment:3 Changed 3 years ago by Tim Graham

At 1aa3e09c2043c88a760e8b73fb95dc8f1ffef50e, the attached test for Django's test suite starts failing with "AttributeError: 'str' object has no attribute '_meta'", then the "relation does not exist" traceback starts happening at fb48eb05816b1ac87d58696cdfe48be18c901f16.

Changed 3 years ago by Tim Graham

Attachment: 24725-test.diff added

comment:4 Changed 3 years ago by Tim Graham

Summary: Bug in Django 1.8 Migrations (ProgrammingError: relation does not exist)"ProgrammingError: relation does not exist" when renaming many-to-many target model

comment:5 Changed 3 years ago by Markus Holtermann

Needs documentation: set
Needs tests: set
Owner: changed from nobody to Markus Holtermann
Status: newassigned

I localized the error to these four lines: https://github.com/django/django/blob/master/django/db/migrations/operations/models.py#L162-L165

They wrongly filter out <ManyToManyRel: debug.john> on the model debug.Spam (i.e. __fake__.Spam):

ipdb> model._meta.get_fields(include_hidden=True)
(<ManyToOneRel: debug.john_consumed_food>, <ManyToManyRel: debug.john>, <django.db.models.fields.AutoField: id>, <django.db.models.fields.CharField: description>)

ipdb> field = model._meta.get_fields(include_hidden=True)[0]
ipdb> field
<ManyToOneRel: debug.john_consumed_food>
ipdb> field.auto_created
True
ipdb> field.concrete
False
ipdb> field.hidden
True
ipdb> field.many_to_many
False

ipdb> field = model._meta.get_fields(include_hidden=True)[1]
ipdb> field
<ManyToManyRel: debug.john>
ipdb> field.auto_created
True
ipdb> field.concrete
False
ipdb> field.hidden
False
ipdb> field.many_to_many
True

comment:6 Changed 3 years ago by Markus Holtermann

Has patch: set
Needs documentation: unset
Needs tests: unset

comment:7 Changed 3 years ago by Tim Graham

Triage Stage: AcceptedReady for checkin

comment:8 Changed 3 years ago by Markus Holtermann <info@…>

Resolution: fixed
Status: assignedclosed

In 63f9b633:

Fixed #24725 -- Allowed renaming of target models in ManyToMany relations

This is a regression caused by introducing rendered migration states in
1aa3e09c2043 and the _meta refactoring in fb48eb05816b.

Thanks to Danilo Bargen for reporting the issue and Marten Kenbeek and
Tim Graham for triaging the bug and providing the initial test case.

comment:9 Changed 3 years ago by Danilo Bargen

Brilliant, thanks for the great work and the quick reaction :)

Any idea when a 1.8.1 release can be expected?

comment:10 Changed 3 years ago by Markus Holtermann <info@…>

In bf4dc573:

[1.8.x] Fixed #24725 -- Allowed renaming of target models in ManyToMany relations

This is a regression caused by introducing rendered migration states in
1aa3e09c2043 and the _meta refactoring in fb48eb05816b.

Thanks to Danilo Bargen for reporting the issue and Marten Kenbeek and
Tim Graham for triaging the bug and providing the initial test case.

Backport of 63f9b633f9cb620a4c4764b47dca127706a8d7b4 from master

comment:11 in reply to:  9 Changed 3 years ago by Markus Holtermann

Replying to dbrgn:

Brilliant, thanks for the great work and the quick reaction :)

Any idea when a 1.8.1 release can be expected?

You're welcome. If nothing unexpected happens 1.8.1 is scheduled for tomorrow, iirc.

Note: See TracTickets for help on using tickets.
Back to Top