Opened 5 weeks ago

Closed 4 weeks ago

Last modified 4 weeks ago

#36643 closed New feature (wontfix)

Migrate should not check for consistent history when faking migrations

Reported by: Alexandru Chirila Owned by:
Component: Migrations Version: 5.2
Severity: Normal Keywords:
Cc: Alexandru Chirila Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

If for some reason or another you end up needing to fake apply a migration, it will still run the check_consistent_history.

So if you ended up by accident (or on purpose somehow?) with a invalid migration history, there is no way of fixing it except manually doing things in the django_migrations table.

It seems like a scenario like this ought to be fixable using the "fake" apply of migrations, but as far as I can tell there is no way of doing that as the check will always fell and throw an error, so the fake aplication can never be done.

.venv/lib/python3.12/site-packages/django/db/migrations/loader.py", line 327, in check_consistent_history
    raise InconsistentMigrationHistory(
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration shortner.0001_initial is applied before its dependency charts.0001_initial on database 'default'.

Change History (8)

comment:1 by Andrew Williamson, 5 weeks ago

Hi, I'm struggling to think of the scenario where you'd fake a migration over an inconsistent history, and it'd leave a consistent state that you'd not need to fake the next time. I.e. it'd always be inconsistent, every time. Can you provide an example of where this would help?

in reply to:  1 comment:2 by Alexandru Chirila, 5 weeks ago

Replying to Andrew Williamson:

Hi, I'm struggling to think of the scenario where you'd fake a migration over an inconsistent history, and it'd leave a consistent state that you'd not need to fake the next time. I.e. it'd always be inconsistent, every time. Can you provide an example of where this would help?

Sure, here's what happend to me:

  • I had some squashed migrations for some of the django apps in my project.
  • I had incorrectly verified that all instances have indeed been updated to the squashed migrations
  • Considering I thought all instances were updated, I removed the old migrations from the code.
  • When I updated the one instance that did NOT received the squashed migrations before, the manage.py migrate command was run automatically
  • One of the apps did not have any squashed migrations, so it remained marked as applied and appeared updated (i.e. shortner in this case)
  • The same app shortner also was depending on a another app charts that has had squashed migration, and as far as Django is concerned it looked like none of the migrations were applied for charts but all migrations were applied to shortner. That's because, when the migrate script ran it marked all migrations as being unapplied for charts.
  • The consistency check was seeing that migrations for shortner were applied before charts. Even though both apps were up to date will all the schema changes.

Beeing in this scenario, I thought that: "Hey, no problem, the DB itself is up to date. Django has this 'fake' apply migration utility just to bail you out when you mess up massively as I did."

Well, apparently I could not be bailed out from this scenario by the "fake apply" migration because of this "consistency check". So I ended up messing around directly into django internal tables (which I would rather always avoid) to fix my issue because the "migrate" command refused to do anything at all.


Definetly this was a big user error on my part, combined with an unfortunate set of circumstances. That said, I don't think that there is any good reason to perform a consistency check when running the "fake apply", since presumably you are in weird state anyway and you're using the "fake apply" to fix something that was caused by some manual error.

If there is a good reason to run the check in this scenario as well, a simple flag that allows you to disable it would be useful I think. I did try "--skip-checks" but that doesn't work as far as I can tell.

Last edited 5 weeks ago by Alexandru Chirila (previous) (diff)

comment:3 by Augusto Pontes, 5 weeks ago

Hi, good evening, i tried to replicate what you did, and i believe that i came up with a simple solution, that can ignore the check_consistent_history, and basically heres what i did: i searched into thedjango/core/management/commands/migrate.py and django/core/management/commands/makemigrations.py path, and i add this command line here at the add_arguments(on both paths) method:

parser.add_argument(
      '--bypass-consistency-check',
       action='store_true',
        dest='bypass_consistency_check',
        help='Bypass the migration history consistency check. Use with caution.',
   )

and on the handle method at the django/core/management/commands/migrate.py i changed this part:

(line: 124)

# Raise an error if any migrations are applied before their
# dependencies.
   if not options['bypass_consistency_check']:
       executor.loader.check_consistent_history(connection)

at the django/core/management/commands/makemigrations.py i edited this line here, before the verification:

(line: 155)

        if not options['bypass_consistency_check']:
            for alias in sorted(aliases_to_check):
            ....

to test this, you can type on the terminal:

 python makemigrations.py your_app --bypass-consistency-check
 python manage.py migrate your_app your_migration_file --fake --bypass-consistency-check

see if this works

comment:4 by Natalia Bidart, 4 weeks ago

Easy pickings: unset
Resolution: wontfix
Status: newclosed
Type: BugNew feature

Hello! Thank you for taking the time to create this ticket. To me, this is more of a feature request than a bug. The current behavior is intentional: Django always checks for a consistent migration history to protect projects from entering an invalid state, even when using --fake.

The need to bypass this check seems very specific to a niche recovery scenario rather than something that applies broadly. Django aims to provide robust, predictable behavior for common cases, so changing this would fall outside that scope.

Given the above, I'll close the ticket accordingly. If you'd like to pursue this as a potential feature, please review the feature request guidelines for more details.

comment:5 by Natalia Bidart, 4 weeks ago

After some further research, it's worth mentioning that this seems somewhat related to #28250, which focused on handling a specific edge case (--fake-initial) where ignoring the consistency check may be considered safe.

in reply to:  4 comment:6 by Alexandru Chirila, 4 weeks ago

Replying to Natalia Bidart:

The current behavior is intentional: Django always checks for a consistent migration history to protect projects from entering an invalid state, even when using --fake.

I'm not sure I understand, if the consistency check is in place to prevent entering an invalid state, why is it being run before the migration runs? Wouldn't it make sense to run it after the migration is run (but before commit)? Or alternatively it should check consistency in a "dry run" mode, to compute the state of the dependency graph after the the migrate runs. Either of these would cover my case as well, and make the consistency check actually useful.

Because as it stands the consistency checks, does not in fact prevent entering an invalid state. The only thing it achieves is blocking using the migrate script if you are already in an invalid state.

comment:7 by Simon Charette, 4 weeks ago

Because as it stands the consistency checks, does not in fact prevent entering an invalid state. The only thing it achieves is blocking using the migrate script if you are already in an invalid state.

I believe things are this way because the migrate command, with and without --fake, makes graph manipulation expecting the history graph to be in a consistent state prior to making these changes and that the operations it performs could venture into undefined behaviour territory without these per-requisite.

To make a typing analogy, if a piece of code expect a graph to follow specific schema (e.g. single head per app, acyclic) and you force it through the pipe in an unexpected form then it can result in damageable ways.

Given undefined behaviour is not something we want anywhere close to database schema alterations I believe the current approach is reasonable.

Version 0, edited 4 weeks ago by Simon Charette (next)

comment:8 by Alexandru Chirila, 4 weeks ago

While I don't agree with the conclusion here, I do understand where it's coming from, so I'll leave it be.


I'll leave here the hacky solution for this problem, for any unfortunate soul that finds themselves in such a terrible mess that I've created for myself.

  • Check the status of migrations using ./manage.py showmigrations, looks for the ones that are applied and are confusing the Django consistency check.
  • Remove from the DB directly the applied migrations for the app(s) in question:
delete from django_migrations where app='APPNAME';
  • Fake apply migrations for the one just removed:
./manage.py migrate APPNAME --fake
  • Fake apply any other migrations for the other apps that have been squashed
Note: See TracTickets for help on using tickets.
Back to Top