Can't migrate contenttypes and auth to zero

[Florian Apolloner]

Trying to migrate either auth to zero results in:

./manage.py migrate -v3 auth zero
Operations to perform:
  Unapply all migrations: auth
Running pre-migrate handlers for application admin
Running pre-migrate handlers for application auth
Running pre-migrate handlers for application contenttypes
Running pre-migrate handlers for application sessions
Running migrations:
  Unapplying auth.0005_alter_user_last_login_null... OK (0.023s)
  Unapplying auth.0004_alter_user_username_opts... OK (0.013s)
  Unapplying auth.0003_alter_user_email_max_length... OK (0.015s)
  Unapplying auth.0002_alter_permission_name_max_length... OK (0.008s)
  Unapplying admin.0001_initial... OK (0.014s)
  Unapplying auth.0001_initial... OK (0.035s)
Running post-migrate handlers for application admin
Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/florian/sources/django.git/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/home/florian/sources/django.git/django/core/management/__init__.py", line 330, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/florian/sources/django.git/django/core/management/base.py", line 391, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/florian/sources/django.git/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/home/florian/sources/django.git/django/core/management/commands/migrate.py", line 216, in handle
    emit_post_migrate_signal(created_models, self.verbosity, self.interactive, connection.alias)
  File "/home/florian/sources/django.git/django/core/management/sql.py", line 280, in emit_post_migrate_signal
    using=db)
  File "/home/florian/sources/django.git/django/dispatch/dispatcher.py", line 198, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/home/florian/sources/django.git/django/contrib/auth/management/__init__.py", line 93, in create_permissions
    "content_type", "codename"
  File "/home/florian/sources/django.git/django/db/models/query.py", line 164, in __iter__
    self._fetch_all()
  File "/home/florian/sources/django.git/django/db/models/query.py", line 999, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/home/florian/sources/django.git/django/db/models/query.py", line 1254, in iterator
    for row in self.query.get_compiler(self.db).results_iter():
  File "/home/florian/sources/django.git/django/db/models/sql/compiler.py", line 747, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/home/florian/sources/django.git/django/db/models/sql/compiler.py", line 831, in execute_sql
    cursor.execute(sql, params)
  File "/home/florian/sources/django.git/django/db/backends/utils.py", line 80, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/home/florian/sources/django.git/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/home/florian/sources/django.git/django/db/utils.py", line 95, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/florian/sources/django.git/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "auth_permission" does not exist
LINE 1: ...ntent_type_id", "auth_permission"."codename" FROM "auth_perm...

and migrating contenttypes to zero:

./manage.py migrate -v3 contenttypes zero
Operations to perform:
  Unapply all migrations: contenttypes
Running pre-migrate handlers for application admin
Running pre-migrate handlers for application auth
Running pre-migrate handlers for application contenttypes
Running pre-migrate handlers for application sessions
Running migrations:
  Unapplying contenttypes.0001_initial... OK (0.029s)
Running post-migrate handlers for application admin
Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/florian/sources/django.git/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/home/florian/sources/django.git/django/core/management/__init__.py", line 330, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/florian/sources/django.git/django/core/management/base.py", line 391, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/florian/sources/django.git/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/home/florian/sources/django.git/django/core/management/commands/migrate.py", line 216, in handle
    emit_post_migrate_signal(created_models, self.verbosity, self.interactive, connection.alias)
  File "/home/florian/sources/django.git/django/core/management/sql.py", line 280, in emit_post_migrate_signal
    using=db)
  File "/home/florian/sources/django.git/django/dispatch/dispatcher.py", line 198, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/home/florian/sources/django.git/django/contrib/auth/management/__init__.py", line 82, in create_permissions
    ctype = ContentType.objects.db_manager(using).get_for_model(klass)
  File "/home/florian/sources/django.git/django/contrib/contenttypes/models.py", line 60, in get_for_model
    "Error creating new content types. Please make sure contenttypes "
RuntimeError: Error creating new content types. Please make sure contenttypes is migrated before trying to migrate apps individually.

Not really sure how to best fix this, should the signal handlers check the migration table?

[Tim Graham]

If it's feasible, an easy way to introspect whether a model's table exists seems useful (not sure if that solution could be backported to 1.7 though).

#22411 is where that error message was added, so maybe that fix could be modified as well if we can do better.

[Tim Graham]

See the comments of #24067 for a proposed solution that would allow addressing this.

[Simon Charette]

Created #24100 to track the new feature required for the proposed solution in #24067.

[Markus Holtermann]

I created an initial PR that tackles the problem until #24100 is fixed.

​https://github.com/django/django/pull/3894

[Markus Holtermann <info@…>]

In 737d24923ac69bb8b89af1bb2f3f4c4c744349e8:

Fixed #24075 -- Prevented running post_migrate signals when unapplying initial migrations of contenttypes and auth

Thanks Florian Apolloner for the report and Claude Paroz and Tim Graham for the review and help on the patch.

[Markus Holtermann <info@…>]

In 478546fcef38d95866a92bc44d10e15b26c7254c:

[1.7.x] Fixed #24075 -- Prevented running post_migrate signals when unapplying initial migrations of contenttypes and auth

Thanks Florian Apolloner for the report and Claude Paroz and Tim Graham for the review and help on the patch.

Backport of 737d24923ac69bb8b89af1bb2f3f4c4c744349e8 from master.

[Markus Holtermann <info@…>]

In 51dc617b21e67636d96cf645905797a4d6ff4bf0:

Refs #24075 -- Silenced needless call_command output while running tests

Thanks Tim Graham for the report

[Markus Holtermann <info@…>]

In b419bd38431b83eec93376cd911e2b17eb8e7342:

[1.7.x] Refs #24075 -- Silenced needless call_command output while running tests

Thanks Tim Graham for the report

Backport of 51dc617b21e67636d96cf645905797a4d6ff4bf0 from master

[Markus Holtermann]

I'm reverting these patches (​https://github.com/django/django/pull/4079) as they cause more harm (severe impact on test performance (#24251) and problems in multi-database setups (#24298)) than good. The described error most likely only occurs on development systems where dropping the respective tables or the entire database shouldn't be a problem.

This issue can be fixed with #24100 instead.

[Markus Holtermann <info@…>]

In bd3d796ecd9a66832ad26024df65caeb63b60a5d:

Revert "Refs #24075 -- Silenced needless call_command output while running tests"

This reverts commit 51dc617b21e67636d96cf645905797a4d6ff4bf0.

[Markus Holtermann <info@…>]

In 2832a9b028c267997b2fd3dd0989670d57cdd08f:

Revert "Fixed #24075 -- Prevented running post_migrate signals when unapplying initial migrations of contenttypes and auth"

This reverts commit 737d24923ac69bb8b89af1bb2f3f4c4c744349e8.

[Markus Holtermann <info@…>]

In b2b5ea88b7159a89e373a71f215596de4d8b015f:

[1.8.x] Revert "Refs #24075 -- Silenced needless call_command output while running tests"

This reverts commit 51dc617b21e67636d96cf645905797a4d6ff4bf0.

Backport of bd3d796ecd9a66832ad26024df65caeb63b60a5d from master

[Markus Holtermann <info@…>]

In edbf6de7536f7a6c1e5df019a5e1947d2c9dadf8:

[1.8.x] Revert "Fixed #24075 -- Prevented running post_migrate signals when unapplying initial migrations of contenttypes and auth"

This reverts commit 737d24923ac69bb8b89af1bb2f3f4c4c744349e8.

Backport of 2832a9b028c267997b2fd3dd0989670d57cdd08f from master

[Markus Holtermann <info@…>]

In e6ffe43b670dddafd97360f81441e9e590d3408b:

[1.7.x] Revert "[1.7.x] Refs #24075 -- Silenced needless call_command output while running tests"

This reverts commit b419bd38431b83eec93376cd911e2b17eb8e7342.

Backport of bd3d796ecd9a66832ad26024df65caeb63b60a5d from master

[Markus Holtermann <info@…>]

In 208d5c42e7d0f8ceea002542ba4b2eaa5dd6dca7:

[1.7.x] Revert "[1.7.x] Fixed #24075 -- Prevented running post_migrate signals when unapplying initial migrations of contenttypes and auth"

This reverts commit 478546fcef38d95866a92bc44d10e15b26c7254c.

Backport of 2832a9b028c267997b2fd3dd0989670d57cdd08f from master

[Simon Charette]

I have a mostly working ​PR that is still missing tests and requires some MigrationExecutor.migration_plan() tweaking that I'll try to figure out with the help of Markus.

[Simon Charette]

The proposed patch now allows the auth, contenttypes and sites application to be migrated to any migration (zero included).

I'm still trying to figure out how to test these changes as the test suite disables migrations for the auth and contenttypes application (using MIGRATION_MODULES['app'] = None).

[Simon Charette]

I managed to add tests for the auth, contenttypes and sites applications. This should be ready for a final review.

[Simon Charette <charette.s@…>]

In 61a16e02:

Fixed #24075 -- Used post-migration models in contrib apps receivers.

Thanks Markus and Tim for the review.