Add --dry-run to migrate

[Richard Eames]

In a similar vein to https://code.djangoproject.com/ticket/23263, I would like to propose that a "--dry-run" or "--check" be added to the migrate command. Basically, I want to be able to run through the migrations without it making database changes, so that I can see if I've broken something, especially if I'm writing a migration with a RunPython.

Another way I can see this working, is if sqlmigrate could also output all the sql that was issued, including those issued by RunPython, so that if I had the following migration [0] it would output something similar to [1]

[0] Migration:

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations

def update_contenttypes(apps, schema_editor):
        ContentType = apps.get_models('contenttypes', 'ContentType')
        ContentType.objects.get(pk = 1)

class Migration(migrations.Migration):
        dependencies = [
                ('everdeal', '0001_initial'),
        ]
        
        operations = [
                migrations.RunPython(update_contenttypes)
        ]

[1] Output:

BEGIN;
--
-- MIGRATION NOW PERFORMS OPERATION THAT CANNOT BE WRITTEN AS SQL:
-- Raw Python operation
SELECT * FROM contenttypes_contenttype WHERE id=1;
--

ROLLBACK;

[Tim Graham]

I don't see a way to implement what you propose. I don't think there is a way to output SQL for arbitrary RunPython without actually running it as queries may dependent on each other (for example manipulating models in a loop of MyModel.objects.all()). If you have an idea, please provide it (or a patch).

[Richard Eames]

Maybe my understanding of databases is wrong, but couldn't you create a savepoint at the beginning, do all the migrations - including the RunPython - then issue a rollback at the end?

[Simon Charette]

@Naddiseo unfortunately not all supported backends support transactional DDL which might end up committing the active transaction silently.

We can't assume only SQL will be executed by RunPython given the provided schema_editor.

[Tim Graham]

(Was about to make a similar comment as @charettes.)

As for alternatives: I don't quite understand your use case "so I can see if I've broken something." Migrations are executed when you run tests so you could test it there. Also the migration is executed in a transaction, so if it breaks somewhere in the middle, any changes should be rolledback. Maybe you could elaborate why either of these options aren't sufficient.

[Richard Eames]

Also the migration is executed in a transaction, so if it breaks somewhere in the middle, any changes should be rolledback.

Unless it's a backend that doesn't support transactions?

My "use case" is that I like to do incremental testing while I'm developing - and by testing, I don't mean test cases, I mean running the code to see if it doesn't have runtime errors - so if I'm writing a data migration, I may right a small section of it, then run the migration to see if what I have works, then move on to the next part, but I don't want the database to be changed when I run it again. Currently, I can work around this by raising an Exception to force a rollback; providing a "dry-run" option would, for me, be a preferable approach, and providing the would-be committed sql would aid in any kind of debugging where looking at the sql is necessary.

I guess if not all of the backends support transactions, then I guess this can't really be done (cleanly anyway), and this ticket can be resolved. Thanks for your time.

[Alexander Schepanovski]

This would be use full even without RunPython support. I used this feature a lot in south.

[Markus Holtermann]

If you want to see the particular SQL code generated for a migration you can use ​sqlmigrate

[Craig de Stigter]

I've re-requested a similar thing in #29198

[Michael]

The --plan option does not help me, it just says Raw Python operation for my custom migration.

Maybe this will help others: I just added a error at the end of the migration, so that it rolls back, now can run it as many times as desired, then remove the error when confident:

from django.db import migrations


def forwards_func(apps, schema_editor):
    Meeting = apps.get_model("meetings", "Meeting")
    db_alias = schema_editor.connection.alias
    for meeting in Meeting.objects.using(db_alias).all():
        if meeting.meeting_utc is None:
            print(meeting.meeting_date, meeting.meeting_time)
    raise ValueError("Don't commit!")  # <----------------

class Migration(migrations.Migration):
    dependencies = [
        ('meetings', '0004_meeting_meeting_utc_meeting_time_is_null'),
    ]

    operations = [
        migrations.RunPython(forwards_func),
    ]