Opened 6 years ago

Closed 4 years ago

#24075 closed Bug (fixed)

Can't migrate contenttypes and auth to zero

Reported by: Florian Apolloner Owned by: nobody
Component: Migrations Version: master
Severity: Normal Keywords:
Cc: Florian Apolloner, Markus Holtermann Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Trying to migrate either auth to zero results in:

./manage.py migrate -v3 auth zero
Operations to perform:
  Unapply all migrations: auth
Running pre-migrate handlers for application admin
Running pre-migrate handlers for application auth
Running pre-migrate handlers for application contenttypes
Running pre-migrate handlers for application sessions
Running migrations:
  Unapplying auth.0005_alter_user_last_login_null... OK (0.023s)
  Unapplying auth.0004_alter_user_username_opts... OK (0.013s)
  Unapplying auth.0003_alter_user_email_max_length... OK (0.015s)
  Unapplying auth.0002_alter_permission_name_max_length... OK (0.008s)
  Unapplying admin.0001_initial... OK (0.014s)
  Unapplying auth.0001_initial... OK (0.035s)
Running post-migrate handlers for application admin
Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/florian/sources/django.git/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/home/florian/sources/django.git/django/core/management/__init__.py", line 330, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/florian/sources/django.git/django/core/management/base.py", line 391, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/florian/sources/django.git/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/home/florian/sources/django.git/django/core/management/commands/migrate.py", line 216, in handle
    emit_post_migrate_signal(created_models, self.verbosity, self.interactive, connection.alias)
  File "/home/florian/sources/django.git/django/core/management/sql.py", line 280, in emit_post_migrate_signal
    using=db)
  File "/home/florian/sources/django.git/django/dispatch/dispatcher.py", line 198, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/home/florian/sources/django.git/django/contrib/auth/management/__init__.py", line 93, in create_permissions
    "content_type", "codename"
  File "/home/florian/sources/django.git/django/db/models/query.py", line 164, in __iter__
    self._fetch_all()
  File "/home/florian/sources/django.git/django/db/models/query.py", line 999, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/home/florian/sources/django.git/django/db/models/query.py", line 1254, in iterator
    for row in self.query.get_compiler(self.db).results_iter():
  File "/home/florian/sources/django.git/django/db/models/sql/compiler.py", line 747, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/home/florian/sources/django.git/django/db/models/sql/compiler.py", line 831, in execute_sql
    cursor.execute(sql, params)
  File "/home/florian/sources/django.git/django/db/backends/utils.py", line 80, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/home/florian/sources/django.git/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/home/florian/sources/django.git/django/db/utils.py", line 95, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/florian/sources/django.git/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "auth_permission" does not exist
LINE 1: ...ntent_type_id", "auth_permission"."codename" FROM "auth_perm...

and migrating contenttypes to zero:

./manage.py migrate -v3 contenttypes zero
Operations to perform:
  Unapply all migrations: contenttypes
Running pre-migrate handlers for application admin
Running pre-migrate handlers for application auth
Running pre-migrate handlers for application contenttypes
Running pre-migrate handlers for application sessions
Running migrations:
  Unapplying contenttypes.0001_initial... OK (0.029s)
Running post-migrate handlers for application admin
Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/florian/sources/django.git/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/home/florian/sources/django.git/django/core/management/__init__.py", line 330, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/florian/sources/django.git/django/core/management/base.py", line 391, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/florian/sources/django.git/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/home/florian/sources/django.git/django/core/management/commands/migrate.py", line 216, in handle
    emit_post_migrate_signal(created_models, self.verbosity, self.interactive, connection.alias)
  File "/home/florian/sources/django.git/django/core/management/sql.py", line 280, in emit_post_migrate_signal
    using=db)
  File "/home/florian/sources/django.git/django/dispatch/dispatcher.py", line 198, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/home/florian/sources/django.git/django/contrib/auth/management/__init__.py", line 82, in create_permissions
    ctype = ContentType.objects.db_manager(using).get_for_model(klass)
  File "/home/florian/sources/django.git/django/contrib/contenttypes/models.py", line 60, in get_for_model
    "Error creating new content types. Please make sure contenttypes "
RuntimeError: Error creating new content types. Please make sure contenttypes is migrated before trying to migrate apps individually.

Not really sure how to best fix this, should the signal handlers check the migration table?

Change History (22)

comment:1 Changed 6 years ago by Tim Graham

Triage Stage: UnreviewedAccepted
Version: master1.7

If it's feasible, an easy way to introspect whether a model's table exists seems useful (not sure if that solution could be backported to 1.7 though).

#22411 is where that error message was added, so maybe that fix could be modified as well if we can do better.

comment:2 Changed 6 years ago by Tim Graham

See the comments of #24067 for a proposed solution that would allow addressing this.

comment:3 Changed 6 years ago by Simon Charette

Created #24100 to track the new feature required for the proposed solution in #24067.

comment:4 Changed 6 years ago by Markus Holtermann

Has patch: set
Needs tests: set

I created an initial PR that tackles the problem until #24100 is fixed.

https://github.com/django/django/pull/3894

comment:5 Changed 6 years ago by Tim Graham

Needs tests: unset
Triage Stage: AcceptedReady for checkin

comment:6 Changed 6 years ago by Markus Holtermann <info@…>

Resolution: fixed
Status: newclosed

In 737d24923ac69bb8b89af1bb2f3f4c4c744349e8:

Fixed #24075 -- Prevented running post_migrate signals when unapplying initial migrations of contenttypes and auth

Thanks Florian Apolloner for the report and Claude Paroz and Tim Graham for the review and help on the patch.

comment:7 Changed 6 years ago by Markus Holtermann <info@…>

In 478546fcef38d95866a92bc44d10e15b26c7254c:

[1.7.x] Fixed #24075 -- Prevented running post_migrate signals when unapplying initial migrations of contenttypes and auth

Thanks Florian Apolloner for the report and Claude Paroz and Tim Graham for the review and help on the patch.

Backport of 737d24923ac69bb8b89af1bb2f3f4c4c744349e8 from master.

comment:8 Changed 6 years ago by Markus Holtermann <info@…>

In 51dc617b21e67636d96cf645905797a4d6ff4bf0:

Refs #24075 -- Silenced needless call_command output while running tests

Thanks Tim Graham for the report

comment:9 Changed 6 years ago by Markus Holtermann <info@…>

In b419bd38431b83eec93376cd911e2b17eb8e7342:

[1.7.x] Refs #24075 -- Silenced needless call_command output while running tests

Thanks Tim Graham for the report

Backport of 51dc617b21e67636d96cf645905797a4d6ff4bf0 from master

comment:10 Changed 5 years ago by Markus Holtermann

I'm reverting these patches (https://github.com/django/django/pull/4079) as they cause more harm (severe impact on test performance (#24251) and problems in multi-database setups (#24298)) than good. The described error most likely only occurs on development systems where dropping the respective tables or the entire database shouldn't be a problem.

This issue can be fixed with #24100 instead.

comment:11 Changed 5 years ago by Markus Holtermann <info@…>

In bd3d796ecd9a66832ad26024df65caeb63b60a5d:

Revert "Refs #24075 -- Silenced needless call_command output while running tests"

This reverts commit 51dc617b21e67636d96cf645905797a4d6ff4bf0.

comment:12 Changed 5 years ago by Markus Holtermann <info@…>

In 2832a9b028c267997b2fd3dd0989670d57cdd08f:

Revert "Fixed #24075 -- Prevented running post_migrate signals when unapplying initial migrations of contenttypes and auth"

This reverts commit 737d24923ac69bb8b89af1bb2f3f4c4c744349e8.

comment:13 Changed 5 years ago by Markus Holtermann <info@…>

In b2b5ea88b7159a89e373a71f215596de4d8b015f:

[1.8.x] Revert "Refs #24075 -- Silenced needless call_command output while running tests"

This reverts commit 51dc617b21e67636d96cf645905797a4d6ff4bf0.

Backport of bd3d796ecd9a66832ad26024df65caeb63b60a5d from master

comment:14 Changed 5 years ago by Markus Holtermann <info@…>

In edbf6de7536f7a6c1e5df019a5e1947d2c9dadf8:

[1.8.x] Revert "Fixed #24075 -- Prevented running post_migrate signals when unapplying initial migrations of contenttypes and auth"

This reverts commit 737d24923ac69bb8b89af1bb2f3f4c4c744349e8.

Backport of 2832a9b028c267997b2fd3dd0989670d57cdd08f from master

comment:15 Changed 5 years ago by Markus Holtermann <info@…>

In e6ffe43b670dddafd97360f81441e9e590d3408b:

[1.7.x] Revert "[1.7.x] Refs #24075 -- Silenced needless call_command output while running tests"

This reverts commit b419bd38431b83eec93376cd911e2b17eb8e7342.

Backport of bd3d796ecd9a66832ad26024df65caeb63b60a5d from master

comment:16 Changed 5 years ago by Markus Holtermann <info@…>

In 208d5c42e7d0f8ceea002542ba4b2eaa5dd6dca7:

[1.7.x] Revert "[1.7.x] Fixed #24075 -- Prevented running post_migrate signals when unapplying initial migrations of contenttypes and auth"

This reverts commit 478546fcef38d95866a92bc44d10e15b26c7254c.

Backport of 2832a9b028c267997b2fd3dd0989670d57cdd08f from master

comment:17 Changed 5 years ago by Markus Holtermann

Cc: Markus Holtermann added
Has patch: unset
Resolution: fixed
Severity: Release blockerNormal
Status: closednew
Triage Stage: Ready for checkinAccepted

comment:18 Changed 4 years ago by Simon Charette

Has patch: set
Needs tests: set
Patch needs improvement: set
Version: 1.7master

I have a mostly working PR that is still missing tests and requires some MigrationExecutor.migration_plan() tweaking that I'll try to figure out with the help of Markus.

comment:19 Changed 4 years ago by Simon Charette

Needs tests: unset
Patch needs improvement: unset

The proposed patch now allows the auth, contenttypes and sites application to be migrated to any migration (zero included).

I'm still trying to figure out how to test these changes as the test suite disables migrations for the auth and contenttypes application (using MIGRATION_MODULES['app'] = None).

comment:20 Changed 4 years ago by Simon Charette

I managed to add tests for the auth, contenttypes and sites applications. This should be ready for a final review.

comment:21 Changed 4 years ago by Tim Graham

Triage Stage: AcceptedReady for checkin

comment:22 Changed 4 years ago by Simon Charette <charette.s@…>

Resolution: fixed
Status: newclosed

In 61a16e02:

Fixed #24075 -- Used post-migration models in contrib apps receivers.

Thanks Markus and Tim for the review.

Note: See TracTickets for help on using tickets.
Back to Top