Generated migration file is not detected by django because of the name of newly generated migration file

[Bishal Gautam]

After a new project is created we run:
python manage.py migrate to sync with our auth database models.

Lets create an app called myapp, register this app on settings. Now we create a model inside myapp as:

class MyModel(models.Model):
    name = models.CharField(max_length=100)
    age = models.IntegerField()

Lets run makemigrations and migrate command for the app. This will generate a migration file with name 0001_initial.py inside myapp/migrations/ folder.

Now let us create a constraints for the model as:

class MyModel(models.Model):
    name = models.CharField(max_length=100)
    age = models.IntegerField()

    class Meta:
        constraints = [
            models.CheckConstraint(
                check=models.Q(age__gte=1),
                name="Age should not be.less.than.one."
            )
        ]

When we added CheckConstraint for our previous model, we need to run makemigrations and migrate to reflect the new changes on database.
Running makemigrations will generate a migration file inside user/migrations/ with name 0002_mymodel_age should not be.less.than.one..py. When we try to run migrate, django will not be able to detect this file and NO changes will be applied to existing database. If we run makemigrations again, the same file will be generated again. No matter how many time we run makemigrations, the same file will be generated again and again. The newly generated migration file is not detected by django migrate and showmigrations command. This is because of the name of migration file. The new migration file name contains serveral dots and because of this name, migrate/showmigration is not able to find the file.

There are mutiple solutions for this:

• First: Renaming the generated migration file. Remove . (dot) (excluding the .py)from the newly generated migration file. This will make sure that the file is recognized as python file by django migrate/showmigrations command.

• Second: We can change the name of our constraint name inside the migration as :

class Meta:
        constraints = [
            models.CheckConstraint(
                check=models.Q(age__gte=1),
                name="Age should not be less than one"
            )
        ]

The only difference between the constraint that was above and the constraint that is implemented now is in the value of name parameter for models.Q(). We removed the exisiting "dot" . from the name. Running makemigration will generate a migration file with name 0002_mymodel_age should not be less than one.py. Now running the migrate/showmigrations command will find the new migration file. This example is also commited in the repository​https://github.com/bisalgt/test-django-migrations. One can look at the commit history and checkout to proper stage as needed , described by commit message on the repository commit history.

• Third: I have replaced the name of the migration file in django source code so that .(dot) gets replaced by some character_. I am also attaching the associated pull request ​https://github.com/django/django/pull/16080.

Using third approach makes sure that no one has to go through the step First and Second as described above. The solution would work out of the box.

[David Sanders]

Can confirm this.

It's worth highlighting is that migrations literally copies the constraint name verbatim for any invalid Python module character not just dots "." and will happily apply these (as long as there's no dots) – as long as we're not doing any imports of migrations we should be fine.

The PR replaces the dots with underscores - that's one solution which addresses the immediate issue but I'm just wondering whether:

a.) Any invalid character is replaced to ensure there are no more problems like this; or
b.) Migrations (ie showmigrations and migrate) do not look skip files with multiple dots?

[Bishal Gautam]

Invalid character in constraint name leads to invalid file so django is unable to create new migrations file

[Bishal Gautam]

Yes, as an experiment, I changed the name of constraint by putting * in the constraint name. Django created migration file name with the exact name of the constraint but failed to create a migration file because of invalid name of the file. I have also added an attachment for this. Kindly find above.

• This didnot create a migration file and generated a error from django core. I think it still tells the user that the invalid file name causes this issue.
• In case of dot . , migration file was created everytime. Generated migration file was getting replaced by new one. Django was not able to find the migration file using migrate command. So I think this dot character error is a bit misleading to users and needs to be replaced.

Or may be we can replace most of the frequently used invalid character if replacing doesnot compromise the performance of project.

[David Sanders]

Nice detective work Bishal :)

It's probably worth waiting for a member of the triage team to make a decision on whether to replace all non-valid Python identifier chars or special filesystem chars (or maybe even decide to do something else)

[Bishal Gautam]

Thank you so much David.

Looking forward to hear from the triage team.

Regards,
Bishal Gautam

[Claude Paroz]

I think replacing any invalid character is the way to go.

[Mariusz Felisiak]

​PR

Regression in fa58450a9ab8a1bdd2a5090b51b00078fd85ffa6.

[Adam Johnson]

I’ve made an alternative PR: ​https://github.com/django/django/pull/16117

[Bishal Gautam]

Looks great Adam. Thank you Team for acknowledging this Ticket. Looking forward to contribute more in future.

Regards,
Bishal Gautam

[Mariusz Felisiak <felisiak.mariusz@…>]

In cd03e8e:

Fixed #34050 -- Replaced invalid chars in migration names with '_'.

Thanks to Bishal Gautam for the report and initial implementation.

Regression in fa58450a9ab8a1bdd2a5090b51b00078fd85ffa6.

Co-Authored-By: Bishal Gautam <bisalgt@…>