Opened 9 years ago
Last modified 8 months ago
#26407 new Cleanup/optimization
Investigate applying transitive reduction to migration graph.
Reported by: | Jarek Glowacki | Owned by: | nobody |
---|---|---|---|
Component: | Migrations | Version: | dev |
Severity: | Normal | Keywords: | transitive reduction showmigrations |
Cc: | Markus Holtermann, emorley@…, Ülgen Sarıkavak | Triage Stage: | Accepted |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
This stemmed from a discussion between @markush and myself in https://github.com/django/django/pull/6254.
We should investigate migration loading performance for the following:
- Requiring the migration graph to always have the minimum number of edges. This would mean enforcing transitive reduction whenever a new edge is added. This would be a trade off between the additional computation to ensure the graph is minimal versus the reduced computation in traversing the graph for ancestors/descendants/etc.
- Performing a single transitive reduction step once the graph has been built. Again, this would be to simplify any later traversal.
Closely related to this is the showmigrations
command, with the --plan
flag at verbosity 2. This adds a lists of migration dependencies in brackets next to every migration in the plan. Currently each list of dependencies is constructed from the original Migration
object's dependencies
list, with:
- appropriate replacements made for migrations that have been replaced by squash migrations, and
- additional dependencies added based on any
run_before
declarations.
So what this shows is the list of dependencies, somewhat rawly, as set by the user in their migration files. If the user declares redundant dependencies, then these bleed into the migration graph, which propagates to the list of dependencies printed by showmigrations
. The question is what do we want it to show? This raw list may help the user with debugging, but it could get cluttered if there is an abundance of superfluous dependencies. So what we could do is print dependencies that come from the migration graph after it has been transitively reduced. This would improve the brevity, though whether it would be helpful to the user is up for debate. We could also try using both reduced and unreduced migration graphs to emphasise superfluous dependencies to the user.
TLDR:
1) Investigate migration performance with transitive reduction:
- at every edge addition
- after graph is built
2) Decide what would be most useful to list for showmigrations --plan -v2
:
- [current] Raw dependencies (with changes to account for replacements and
run_before
). - Only critical dependencies (by using reduced graph).
- Raw dependencies with superfluous edges somehow emphasised.
To be clear, I don't see a problem with the current system, but thought this might be worth investigating/discussing.
Change History (4)
comment:1 by , 9 years ago
Triage Stage: | Unreviewed → Accepted |
---|
comment:3 by , 8 years ago
Cc: | added |
---|
comment:4 by , 8 months ago
Cc: | added |
---|
The following simple implementation does not scale for more than ~ 50 to 100 nodes:
django/db/migrations/graph.py