Opened 8 years ago
Closed 3 years ago
#26760 closed Cleanup/optimization (fixed)
Delete nonexistent migrations from django_migrations table
Reported by: | Jarek Glowacki | Owned by: | Jacob Walls |
---|---|---|---|
Component: | Migrations | Version: | dev |
Severity: | Normal | Keywords: | django_migrations squash migrations |
Cc: | Markus Holtermann, dev@… | Triage Stage: | Ready for checkin |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
Django adds a django_migrations
table to the database which list all of the migrations that have been applied (and when).
With the introduction of squashmigrations
, it is possible for this table to contain a lot of old migrations that no longer exist. This can be problematic if naming duplication occurs:
Example:
I have an app with:
my_app/migrations/ 0001_initial.py 0002_blah.py 0003_blah.py
I squash and delete replaced migrations:
my_app/migrations/ 0001_initial_squashed_0003_blah.py
I create a new migration and use poor naming:
my_app/migrations/ 0001_initial_squashed_0003_blah.py 0002_blah.py
My new migration never runs because the django_migrations
table thinks it has already been applied.
I propose truncation of the django_migrations
table so that it includes only migrations that actually exist in the django project. This could be done automatically (when executor runs, or inside the migrate
mcommand). Or have its own mcommand that requires it be run manually. I prefer the automatic approach though.
Pros:
- Cleans up old data that just bloats the database.
- Protects users from the trap mentioned above where a new migration is created with the same name as one that was applied in the past.
Cons:
- A loss of historical information.
Note:
Need to be careful with implementation to avoid a possible new trap if a user squashes migrations and then proceeds to delete the replaced migrations before running the squashed migrations on their database -> django will think squashed migrations haven't been applied and will attempt to reapply them. This can be remedied simply by not removing migrations mentioned in replaces
lists of other migrations from django_migrations
(ie. we'd consider replaced migrations as still existing, even if their actual files have already been removed).
Change History (14)
comment:1 by , 8 years ago
Summary: | django_migrations table is never cleaned. → Delete nonexistent migrations from django_migrations table |
---|---|
Triage Stage: | Unreviewed → Accepted |
comment:2 by , 8 years ago
Note #25255 and #24900 - people sometimes still want to use the squashed migrations (e.g. migrate back into the series that was squashed) in the presence of the merged migration. I note that the suggestion is only to remove from the database migrations whose files no longer exist. This plays well with the suggestion to keep the migrations named in "replaces", as we recommend that when the squashed migration files are removed, the "replaces" clause is also removed from the migration.
comment:3 by , 8 years ago
I think there is no need to delete records from db because they are almost always generated automatically and have unique names.
I see one issue with automatic deleting of records: if I have django-apps which are temporary disabled or are enabled and deploy in this moment, data about their migrations will be lost.
comment:4 by , 8 years ago
Cc: | added |
---|
comment:5 by , 3 years ago
Has patch: | set |
---|---|
Owner: | changed from | to
Patch needs improvement: | set |
Status: | new → assigned |
Working on some test cleanup/isolation issues I just discovered.
comment:6 by , 3 years ago
Patch needs improvement: | unset |
---|
comment:7 by , 3 years ago
Patch needs improvement: | set |
---|
comment:8 by , 3 years ago
Patch needs improvement: | unset |
---|
comment:9 by , 3 years ago
Patch needs improvement: | set |
---|
comment:10 by , 3 years ago
Patch needs improvement: | unset |
---|
comment:11 by , 3 years ago
Patch needs improvement: | set |
---|
comment:12 by , 3 years ago
Patch needs improvement: | unset |
---|
comment:13 by , 3 years ago
Triage Stage: | Accepted → Ready for checkin |
---|
In #26429 we added a timestamp to merge migration names to reduce the likelihood of collisions there. I acknowledge this could happen in other situations though.