Previously squashed migration fails under Python 3 with "multiple leaf nodes" error

[Ed Morley]

Context

Whilst Django 2.0 / master have dropped support for Python 2, for many existing Python 2.7 projects the common path to the ideal "Django 2 + Python 3" end state will be an interim step of "Django 1.11 + Python 3". As such, it seems in the best interests of making that transition as easy as possible, to reduce the number of people left behind on older Django versions.

STR

1. Clone ​testcase-django-squashmigrations-py3compat
2. mkvirtualenv dj111-py2 -p python2.7
3. pip install Django==1.11.12
4. ./manage.py squashmigrations testapp 0002 --noinput
5. ./manage.py migrate
6. mkvirtualenv dj111-py3 -p python3.6
7. pip install Django==1.11.12
8. ./manage.py migrate

Expected

The migrate command at step 8 completes successfully, or else a less misleading error message shown.

Actual

Step 8 results in:

CommandError: Conflicting migrations detected; multiple leaf nodes in the migration graph:
(0002_some_change, 0001_squashed_0002_some_change in testapp).
To fix them run 'python manage.py makemigrations --merge'

Running the suggested manage.py makemigrations --merge fails too (and it shouldn't be necessary anyway):

Traceback (most recent call last):
  ...
  File ".../django/core/management/commands/makemigrations.py", line 272, in handle_merge
    raise ValueError("Could not find common ancestor of %s" % migration_names)
ValueError: Could not find common ancestor of {'0001_squashed_0002_some_change', '0002_some_change'}

Additional notes

• If the bytestring prefix (b'...') is removed from the migration names specified in replaces in the generated squashed migration, the migration succeeds.
• If prior to squashing the migration, from __future__ import unicode_literals is added to settings.py, then the generated migration does not contain bytestring prefixes. Adding to settings.py is necessary since this project uses the default AppConfig rather than a manually specified one.
• The above was using Python 2.7.14 and Python 3.6.4.
• Related tickets #25906 and #24949, though they seem broader in scope, and don't reference the misleading "multiple leaf nodes" error specifically (which was one of the most confusing aspects when debugging this).

Thoughts on fixes

• Ideally the migrate command's graph traversal would coerce all migration names to string before comparing, which would make the above "just work" even on previously created squashed migrations.
• Alternatively the squashmigration command could coerce the migration names specified in replaces to string before even under Python 2, to ensure the generated migration is correct. However this will still mean existing projects have to make manual edits to existing migrations.
• It might also be worth adding from __future__ import unicode_literals to the startproject template settings.py to catch cases where people don't point at a custom AppConfig.
• Failing all of the above, it would be good to make the error message less misleading and/or mention this case explicitly in the docs. Especially since the current docs suggested workaround of adding unicode_literals and re-running makemigrations doesn't work for squashed migrations (it still results in the multiple leaf nodes error unless the squashed migration is hand-edited). Docs in question:
  • ​https://docs.djangoproject.com/en/1.11/topics/migrations/#supporting-python-2-and-3
  • ​https://docs.djangoproject.com/en/1.11/topics/python3/

Side note: It's not clear whether an app not having a specified AppConfig is fine, or considered legacy/deprecated. If the latter it would be great to clarify this in the docs, and state the disadvantages or even add a deprecation warning.

[Tim Graham]

As far as I see, your proposals may not have much value for versions other than Django 1.11 which is mostly receiving only security and data loss fixes at this time. At least there's now this ticket to document the problem for anyone who searches for it.

[Ed Morley]

Agreed.

Perhaps the best fix would be to update ​this page to recommend manually removing the bytestring attributes in existing migrations (after adding from __future__ import unicode_literals everywhere) rather than running makemigrations?

[Tim Graham]

Generally, we don't update (except for critical fixes) the non-stable versions of the documentation to avoid introducing untranslated text into the translated documentation. I'm not sure if this is worth an exception.

[Tim Graham]

I guess I'd be willing to review a documentation patch if you want to provide it.