Allow migrations directories without __init__.py files

[Benjy Weinberger]

Background: In python 3 a package with no __init__.py is implicitly a namespace package, so it has no __file__ attribute.

The migrate command currently checks for existence of a __file__ attribute on the migrations package. This check was introduced in #21015, because the __file__ attribute was used in migration file discovery.

However, in #23406 migration file discovery was changed to use pkgutil.iter_modules (), instead of direct filesystem access. pkgutil. iter_modules() uses the package's __path__ list, which exists on implicit namespace packages.

As a result, the __file__ check is no longer needed, and in fact prevents migrate from working on namespace packages (implicit or otherwise).

Related work: #29091

[Benjy Weinberger]

Proposed fix in ​https://github.com/django/django/pull/11141
(branch ​https://github.com/benjyw/django/tree/ticket_30300)/

[Carlton Gibson]

Hi Benjy. Thanks for the report — and the test case in the PR! Unless something comes up in review, this seems reasonable to me.

I'm going to call it a new feature, since this behaviour will never have worked I think.

... (implicit or otherwise)

I don't quite follow the or otherwise there?

[Tim Graham]

What's the use case? When makemigrations generates the first migration, it adds an __init__.py file. There was some concern about encouraging namespace packages in ticket:23919#comment:102.

[Benjy Weinberger]

The "or otherwise" refers to namespace packages being either implicit (lacking an init.py) or explicit (an init.py containing an old-style namespace package incantation).

[Benjy Weinberger]

Re the use case:

It seems unnecessary to insist on an empty __init__.py just to preserve a (stale) check for __file__, when they are otherwise not required in Python 3.

And #29091 implies that this should be allowed.

Context is: We've just finished migrating our codebase to all-python 3, and nuked all our __init__py files. We are now upgrading to Django 2.1, and noticing the various places where they're still de-facto required by Django. This particular case seemed unnecessary and easy to fix.

[Václav Řehák]

We had a similar issue with test discovery. One of our developers read an article that __init__.py files are not required on Python3 and started removing them. Everything seemingly worked but some tests were not discovered and did not run in CI (which made it difficult to spot).

I think Django should not required them if technically possible or at least it should be made clear in the documentation (sorry if I overlooked it).

[Benjy Weinberger]

Friendly nudge?

[Mariusz Felisiak]

​PR

[Mariusz Felisiak <felisiak.mariusz@…>]

In 3cd3bebe:

Fixed #30300 -- Allowed migrations to be loaded from directories without init.py file.

[Tim Graham]

I raised this issue ​on the mailing list as there was a thread that concluded that Django shouldn't promote use of implicit namespace packages.

[Tim Graham]

Thus far there haven't been any mailing list replies in favor this change. I created a ​PR to revert it and modify the test accordingly.

[GitHub <noreply@…>]

In ff55adbd:

Reverted "Fixed #30300 -- Allowed migrations to be loaded from directories without init.py file."

This reverts commit 3cd3bebe8921e14b911b36b2a1cbceef8fb6294e.

[Mariusz Felisiak <felisiak.mariusz@…>]

In d004bce:

[3.1.x] Reverted "Fixed #30300 -- Allowed migrations to be loaded from directories without init.py file."

This reverts commit 3cd3bebe8921e14b911b36b2a1cbceef8fb6294e.
Backport of ff55adbd0da6618abaf265d16196bf54f81aa77a from master