Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#33583 closed Bug (invalid)

sqlmigrate crashes on replaces to nonexistent migration.

Reported by: Amin Shah Gilani Owned by: nobody
Component: Migrations Version: 3.2
Severity: Normal Keywords: sqlmigrate, migrate, squash
Cc: David Wobrock Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Amin Shah Gilani)

Hello,

This is my first bug report here so please ask for any additional details you need. I would also appreciate a workaround. If it is obvious, it escapes me.

Summary

When an app is renamed with its migrations including a replaces reference to the old app and migration name (such as by squashing migrations), the behavior between migrate and sqlmigrate becomes inconsistent. migrate happily applies all migrations, but sqlmigrate reports an error.

Other information:

  • This bug was first observed in v3.2, but my demo shows it also affects v4.0.3.
  • This bug affects installations using the social-app-django package, which renamed its app multiple times,
  • I have a demo available here.

Steps to reproduce:

Step 1: Create a migration file, in an app and then rename the app and migration. This will result in a migration similar to this:

# alpha/migrations/0001_initial.py
class Migration(migrations.Migration):

    replaces = [
        ('first', '0001_initial'),
    ]


Step 2: Ensure you have migrations in another app, with the first migration the second app referencing the old migration:

class Migration(migrations.Migration):

    dependencies = [
        ('first', '0001_initial'),
    ]

Step 3: Note that migrate will happily apply all migrations:

python3 manage.py migrate
Operations to perform:
  Apply all migrations: admin, alpha, auth, beta, contenttypes, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying alpha.0001_initial... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying beta.0001_initial... OK
  Applying beta.0002_auto_20220318_0245... OK
  Applying beta.0003_auto_20220318_0247... OK
  Applying sessions.0001_initial... OK

Step 4: However, sqlmigrate will raise an error:

python3 manage.py sqlmigrate beta 0003
Traceback (most recent call last):
  File "/Users/amin/sandbox/mysite/manage.py", line 22, in <module>
    main()
  File "/Users/amin/sandbox/mysite/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 440, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 414, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/commands/sqlmigrate.py", line 38, in execute
    return super().execute(*args, **options)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 460, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/commands/sqlmigrate.py", line 46, in handle
    loader = MigrationLoader(connection, replace_migrations=False)
  File "/usr/local/lib/python3.9/site-packages/django/db/migrations/loader.py", line 58, in __init__
    self.build_graph()
  File "/usr/local/lib/python3.9/site-packages/django/db/migrations/loader.py", line 276, in build_graph
    self.graph.validate_consistency()
  File "/usr/local/lib/python3.9/site-packages/django/db/migrations/graph.py", line 198, in validate_consistency
    [n.raise_error() for n in self.node_map.values() if isinstance(n, DummyNode)]
  File "/usr/local/lib/python3.9/site-packages/django/db/migrations/graph.py", line 198, in <listcomp>
    [n.raise_error() for n in self.node_map.values() if isinstance(n, DummyNode)]
  File "/usr/local/lib/python3.9/site-packages/django/db/migrations/graph.py", line 60, in raise_error
    raise NodeNotFoundError(self.error_message, self.key, origin=self.origin)
django.db.migrations.exceptions.NodeNotFoundError: Migration beta.0001_initial dependencies reference nonexistent parent node ('first', '0001_initial')

Note: I have a demo available to try here.

Actual Behavior

sqlmirate raises an error but migrate processes the migrations without a problem

Expected Behavior

Both commands would consistently either raise an error on these migrations, or run without raising an error

Change History (2)

comment:1 by Mariusz Felisiak, 2 years ago

Cc: David Wobrock added
Component: UncategorizedMigrations
Resolution: invalid
Status: newclosed
Summary: Inconsistency: sqlmigrate reports a broken migration but migrate applies themsqlmigrate crashes on replaces to nonexistent migration.

Thanks for the report, this behavior was changed in d88365708c554efe3c786c3be6da1d9de916360f. However, replaces is documented and supported only for squashing migrations so described flow was never officially supported.

in reply to:  1 comment:2 by Amin Shah Gilani, 2 years ago

Description: modified (diff)

Replying to Mariusz Felisiak:

Thanks for the report, this behavior was changed in d88365708c554efe3c786c3be6da1d9de916360f. However, replaces is documented and supported only for squashing migrations so described flow was never officially supported.

Thank you for your response. Would it then be correct to say that the migrate command needs to be updated as well to raise an error? As it stands the behavior is inconsistent. I've added existing and expected behavior to the original ticket in order to clarify this problem.

Additionally, for applications that depend on relocated migrations in third-party packages, will it be fine to update the now outdated pointer to the new location of the migration?

In the context of the example above, is this fine?

dependencies = [
-    ('first', '0001_initial'),
+    ('alpha, '0001_initial'),
]
Last edited 2 years ago by Amin Shah Gilani (previous) (diff)
Note: See TracTickets for help on using tickets.
Back to Top