Opened 3 years ago

Last modified 2 years ago

#24481 assigned New feature

Improve sqlmigrate to be more flexible and allow bypassing migrations on disk

Reported by: pascal chambon Owned by: Andrew Godwin
Component: Core (Management commands) Version: master
Severity: Normal Keywords: migrate migrations sqlall
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

When upgrading a django+djangocms install to latest versions, things got broken (at least one sql table with missing, even though MIGRATION_MODULES was well filled and "manage.py migrate" indicated all was up-to-date).

I wanted to use sqlall and other commands to diff my existing DB with the "theoretical" one, but django refused, indicating that only sqlmigrate and sqlflush were allowed for apps with migrations. I had to redeploy another install from scratch to do the diff.

Wouldn't it be better to issue a warning in these cases, but let the developper use these tools to understand what's wrong, and manually "fake" migrations when stuffs are out of control ?

Change History (8)

comment:1 Changed 3 years ago by Tim Graham

I think those commands can't work like they did before because when there are multiple migrations, I don't think it's feasible to determine what the resulting database schema will look like.

Maybe an idea would be to make the migration_name argument to sqlmigrate optional so that you could do sqlmigrate <app_name> to get the SQL for all of your migrations in an app. I guess we might be able to make app_name optional as well so that sqlmigrate could easily output the SQL for all migrations of all apps.

comment:2 Changed 3 years ago by pascal chambon

Do you mean "sqlmigrate" would output, in sequence, all "alter tables" commands to recreate the final DBs?

I've had a look at django\core\management\sql.py, a function check_for_migrations() is called to abort when some models use migrations, but other than that, nothing seems to prevent generation from sql for migration-enabled models. There is even, farther, code which looks explicitely dedicated to migration handling ("for model in router.get_migratable_models(app_config, connection.alias, include_auto_created=True)").

From what I understand, these sql* commands browse current model objects, and convert them to sql statements, so I guess they shouldn't care about whether these python classes have some "history" behind them. Or am I misunderstanding how all that works?

Last edited 3 years ago by pascal chambon (previous) (diff)

comment:3 Changed 3 years ago by Tim Graham

Triage Stage: UnreviewedAccepted
Type: BugNew feature
Version: 1.7master

It might work. Would be happy to look at a patch but might have to close this if it ends up being confusing or not feasible to implement.

comment:4 Changed 3 years ago by pascal chambon

Looked at the master-branch code, all the code for these commands has disappeared from it, however it'd really be worth to resurrect the "sqlcreate" command only, it's invaluable for quick debugging of migration troubles.

I'll start a thread on the mailing-list to see what's the status about all this.

comment:5 Changed 2 years ago by Andrew Godwin

I've started working on this here: https://github.com/django/django/pull/4729

comment:6 Changed 2 years ago by Markus Holtermann

Summary: Allow use of sql* commands even on apps with migrationsImprove sqlmigrate to be more flexible and allow bypassing migrations on disk

Andrew, Loic, Florian and I came up with the following list of features we want sqlmigrate to support:

  • just one migration
  • list of migration names
  • start / end of migration graph
  • ignore migrations and build straight from models

comment:7 Changed 2 years ago by Markus Holtermann

Owner: changed from nobody to Andrew Godwin

comment:8 Changed 2 years ago by Andrew Godwin

Status: newassigned
Note: See TracTickets for help on using tickets.
Back to Top