Opened 7 months ago

Closed 2 months ago

Last modified 4 weeks ago

#29198 closed New feature (fixed)

Add a --plan option to the migrate command

Reported by: Craig de Stigter Owned by: Calvin DeBoer
Component: Migrations Version: master
Severity: Normal Keywords:
Cc: Calvin DeBoer Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

While migrating a prod database today I found myself feeling uncomfortable.

In a situation like this:

[X] 0023_add_fairy_dust
[X] 0021_apply_beers_to_database
[ ] 0022_magik_stuff
[ ] 0024_merge

... what will happen when I run django-admin migrate app 0023_add_fairy_dust? Will 0021_apply_beers_to_database get unapplied? Will it try to roll forwards to the merge? Will some dependencies from another app get pulled in? Will any tables get dropped? Something Else?*

We really need a way to quickly sanity-check what's going to happen, before actually doing it.

A dry-run option was proposed in #23347 but then everyone missed the forest for the trees, and it was closed because something about RunPython not being able to be sqlified.

There are various ways to do this:

  • I think a simple --plan showing which migrations will be applied (same as the showmigrations option, but for a specific plan, especially for reverse plans) would probably be enough to ease my mind about unapplying beers.
  • A short summary of the operations involved might be nice.
  • Full SQL output might be cool. But it might also be too verbose for a quick sanity check if there are a lot of changes. And as pointed out in the other ticket, it doesn't work for RunPython (but that's not really a good reason to not do it)

I haven't spent much time in newer releases (currently upgrading a large project from 1.8 to 1.11) but AFAICT this still doesn't exist. Correct me if I'm wrong.

(*Answer: nothing happens - it says No migrations to apply.. I don't think it's possible to unapply those beers, but that's a separate issue.)

Change History (13)

comment:1 Changed 7 months ago by Carlton Gibson

Triage Stage: UnreviewedAccepted
Type: UncategorizedNew feature

This seems as if it should be viable, at least at the individual migration level. The migrate command calls MigrationExecutor.migration_plan(), making the plan available prior to running any operations. We could intercept execution there and format the plan for display.

We could expect to list app_name and migration (in order). Users would need to look at the migrations for the exact operations but, if I'm reading the intent here correctly, that would probably be enough.

comment:2 Changed 7 months ago by Carlton Gibson

Version: 2.0master

comment:3 Changed 6 months ago by Sam Handler

I'm very interested in some version of this functionality as well, though my case may be different enough to warrant another ticket. I'd like to see top-level feedback along the lines suggested by Craig in the specific and general migration case (./manage.py migrate and ./manage.py migrate account 0001_initial). I'd also like to be able to run sqlmigrate against all pending migrations so that I can easily know the queries that will be run against the DB. I'm not too knowledgeable about Django's migration internals (yet!) but am glad to take a first stab at this functionality!

Last edited 6 months ago by Sam Handler (previous) (diff)

comment:4 Changed 5 months ago by Calvin DeBoer

Owner: changed from nobody to Calvin DeBoer
Status: newassigned

comment:5 Changed 5 months ago by Calvin DeBoer

Cc: Calvin DeBoer added

comment:6 Changed 5 months ago by Calvin DeBoer

Added a WIP Pull Request that outlines the basic functionality. Main todo is completing the rest of the testing framework / adjusting code based on comments from django core maintainers.

Running a ./manage.py migrate --plan will yield a look at all the migrations to be run (in order), which all the operations (in order), which inclusion on what that migration will do (if it is adequately defined).

Planned operations:
migrations.0001_initial
    Create model Salamander --> No further Detail
    Raw Python operation --> Grows salamander's tail
migrations.0002_second
    Create model Book --> No further Detail
    Raw SQL operation --> SELECT * FROM migrations_book
migrations.0003_third
    Create model Author --> No further Detail
    Raw SQL operation --> SELECT * FROM migrations_author

Thoughts ?

Last edited 5 months ago by Calvin DeBoer (previous) (diff)

comment:7 Changed 5 months ago by Calvin DeBoer

Has patch: set
Needs documentation: set
Needs tests: set

comment:8 Changed 3 months ago by Calvin DeBoer

Needs documentation: unset
Needs tests: unset

Added documentation and tests

comment:9 Changed 3 months ago by Carlton Gibson

Triage Stage: AcceptedReady for checkin

comment:10 Changed 3 months ago by Tim Graham

Patch needs improvement: set
Triage Stage: Ready for checkinAccepted

comment:11 Changed 2 months ago by Tim Graham <timograham@…>

Resolution: fixed
Status: assignedclosed

In 058d33f3:

Fixed #29198 -- Added migrate --plan option.

comment:12 Changed 4 weeks ago by Tim Graham

I found a crash if RunSQL is passed a list rather than a string. PR

comment:13 Changed 4 weeks ago by Tim Graham <timograham@…>

In 495abe00:

Refs #29198 -- Fixed migrate --plan crash if RunSQL uses a list or tuple.

Also fixed test failures if sqlparse isn't installed.

Note: See TracTickets for help on using tickets.
Back to Top