Opened 7 years ago

Closed 3 years ago

#29026 closed New feature (fixed)

Make makemigrations scriptable / script-friendly

Reported by: Chris Jerdonek Owned by: Jacob Walls
Component: Migrations Version: dev
Severity: Normal Keywords: makemigrations, scripting, stderr, stdout
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Currently, the makemigrations management command doesn't lend itself well to scripting. For example, it writes its progress output to stdout rather than stderr. Also, there doesn't appear to be a structured / programmatic way to figure out what files it has created.

My use case is that in my development environment, I'd like to be able to run makemigrations in a Docker container, find out what files were added (e.g. from makemigrations's output), and then copy those files from the Docker container to my development machine so they can be added to source control.

Currently, there doesn't seem to be an easy way to do this. One way, for example, is to manually read makemigrations's output to find out what apps were affected, and then inspect the directories yourself for the new files.

Better, for example, would be if makemigrations could write the paths to the created files to stdout.

Change History (23)

comment:1 by astandley, 7 years ago

The current makemigrations command would seem to allow for everything you have asked.

# Define People and Poll models
>> python -m manage makemigrations
Migrations for 'newspaper':
  newspaper\migrations\0001_initial.py
    - Create model People
Migrations for 'polls':
  polls\migrations\0001_initial.py
    - Create model Poll

# Add new_field to People Model
>> python -m manage makemigrations
Migrations for 'newspaper':
  newspaper\migrations\0002_people_new_field.py
    - Add field new_field to people

Addressing each point:

Output written to stdout rather than stderr.

You can redirect stdout to stderr for the execution of the command if you want the output in stderr. (Using sys in python, or pipes for a shell script)

A structured / programmatic way to figure out what files it has created.

Run a regex on the output of the migration command. Example pattern: r'Migrations for '(?P<app_name>[^']*)':\n (?P<migration_file>[^\n]*)'

Is there a reason this would not meet your needs?

Last edited 7 years ago by astandley (previous) (diff)

comment:2 by Chris Jerdonek, 7 years ago

I don't consider parsing log output with regexes to be structured output or a programmatic API. That seems brittle and unnecessarily complicated, and also would be hard for Django to make backwards compatibility guarantees about.

What I had in mind was something like log output going to stderr, and the paths of the created files going to stdout -- one per line. If something fancier was needed, json could be outputted. With something like that, there wouldn't be any need for regex parsing and the API would be well-defined.

comment:3 by Tim Graham, 7 years ago

It seems complicated. For example, what if makemigrations requires interactive input from the questioner?

comment:4 by Chris Jerdonek, 7 years ago

My original use case was a non-interactive one (inside a container). But again, you raise a good point. Obviously, piping to stdout won't work if interactivity is required (because you'd want user prompts to go to stdout). This is true of any script, not just Django management commands. Other than that, I don't think the changes I've described would hurt things in that case, aside from possibly the "reliability" issue you mentioned here. That though could be addressed by my follow-up comment to yours. If we wanted a fancier solution, the "structured" stdout could be outputted only in non-interactive mode.

comment:5 by Tim Graham, 7 years ago

Triage Stage: UnreviewedAccepted

I'm skeptical, but I guess if you have a patch to propose, we can evaluate it.

comment:6 by Tim Graham, 6 years ago

I closed #29470 as a duplicate.

comment:7 by Amir Hadi, 6 years ago

If you are developing with Docker, why are you not just mounting your development machines source directory into the container, execute makemigrations and then you have the migrations directly on your machine. This way you can save yourself from parsing anything.

comment:8 by Jacob Walls, 3 years ago

Owner: changed from nobody to Jacob Walls
Status: newassigned

comment:9 by Jacob Walls, 3 years ago

Has patch: set
Last edited 3 years ago by Jacob Walls (previous) (diff)

comment:10 by Jacob Walls, 3 years ago

Needs tests: set

Until I remedy the test failure on Windows.

comment:11 by Jacob Walls, 3 years ago

Needs tests: unset
Patch needs improvement: set

I'm going to take a look at implementing Chris's suggested log() method. (See PR)

comment:12 by Jacob Walls, 3 years ago

Patch needs improvement: unset

comment:13 by Jacob Walls, 3 years ago

Patch needs improvement: set

comment:14 by Jacob Walls, 3 years ago

Patch needs improvement: unset

Up to date w/r/t feedback but for commit reorganization and suggestion to put the new log() method in a standalone PR, which I am willing to do!

comment:15 by Jacob Walls, 3 years ago

Got around to reorganizing the commits and opening a first-step refactoring PR to add log() per Chris's suggestion.

If for whatever reason this is not desired, by all means, close the smaller PR and we can just work on one branch. Cheers.

comment:16 by Mariusz Felisiak <felisiak.mariusz@…>, 3 years ago

In f153e92:

Refs #29026 -- Added log() to makemigrations.

comment:17 by Jacob Walls, 3 years ago

Type: Cleanup/optimizationNew feature

comment:18 by Mariusz Felisiak, 3 years ago

Patch needs improvement: set

comment:19 by Jacob Walls, 3 years ago

Patch needs improvement: unset

comment:20 by Mariusz Felisiak <felisiak.mariusz@…>, 3 years ago

In 0ab58c12:

Refs #29026 -- Allowed customizing InteractiveMigrationQuestioner's prompt destination.

Previously, the questioner did not obey the value of stdout provided
to the command.

comment:21 by Mariusz Felisiak, 3 years ago

Patch needs improvement: set

comment:22 by Mariusz Felisiak, 3 years ago

Patch needs improvement: unset
Triage Stage: AcceptedReady for checkin

comment:23 by Mariusz Felisiak <felisiak.mariusz@…>, 3 years ago

Resolution: fixed
Status: assignedclosed

In 6f78cb6b:

Fixed #29026 -- Added --scriptable option to makemigrations.

Note: See TracTickets for help on using tickets.
Back to Top