Loading app fails with Python 3.3 resulting in: TypeError: '_NamespacePath' object does not support indexing

[elbaschid]

I have just created a new Django project using the freshly released Django 1.7a1 with Python 3.3. I've create the project using django-admin.

Project Setup

    manage.py
    project/
       photo/
          models.py
       settings.py
       urls.py
       wsgi.py

I have created a custom app 'photo' that contains a single model 'Photo' and
nothing else. I've added the app in the settings using the old-style dotted
notation without an 'AppConfig'.

The INSTALLED_APPS in 'settings.py':

    INSTALLED_APPS = [
        ...
        'project.photo',
    ]

Excpected Behaviour

The app 'project.photo' is loaded and './manage.py shell' or './manage.py runserver' are running fine.

Actual Behaviour

When I am running './manage.py migrage' or even just './manage.py shell', I am
getting the following error:

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/elbaschid/.virtualenvs/disperser/lib/python3.3/site-packages/django/core/management/__init__.py", line 427, in execute_from_command_line
    utility.execute()
  File "/home/elbaschid/.virtualenvs/disperser/lib/python3.3/site-packages/django/core/management/__init__.py", line 391, in execute
    django.setup()
  File "/home/elbaschid/.virtualenvs/disperser/lib/python3.3/site-packages/django/__init__.py", line 21, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/elbaschid/.virtualenvs/disperser/lib/python3.3/site-packages/django/apps/registry.py", line 84, in populate
    app_config = AppConfig.create(entry)
  File "/home/elbaschid/.virtualenvs/disperser/lib/python3.3/site-packages/django/apps/base.py", line 111, in create
    return cls(entry, module)
  File "/home/elbaschid/.virtualenvs/disperser/lib/python3.3/site-packages/django/apps/base.py", line 43, in __init__
    self.path = upath(app_module.__path__[0])
TypeError: '_NamespacePath' object does not support indexing

Possible Solution

A similar problem has been reported on the pytest project with a fix that has
been accepted:

​https://bitbucket.org/hpk42/pytest/commits/dac4900b78f2

As the commit message there states "Starting with Python 3.3, NamespacePath
passed to importlib hooks seem to have lost the ability to be accessed by
index.". This seems to be the same issue here.

[Baptiste Mispelon]

Hi,

I'm running Python 3.3.3 too but I can't reproduce the error.

How did you install Django?

[Baptiste Mispelon]

Nevermind, I hadn't read your report thoroughly enough (I created the "photo" app with startapp instead of using the same layout as you).

If I remove the __init__.py file from the app's folder (which is a perfectly valid thing to do in Python 3), it triggers the error.

Since apps without an __init__.py file worked with Django 1.6, I'm going to mark this ticket as a release blocker as well.

In the meantime, you should be able to work around the problem simply by adding an empty __init__.py file to project/photo.

Thanks.

[Tim Graham]

We have a ticket open to document that apps cannot be namespace packages (#17304). If I understand correctly, this is no longer the case?

[Tim Graham]

Carl's reply from IRC: "I think the question I raised here still applies. We look for a lot of things relative to the app's package directory; it's not clear how that would be handled with a namespace package. It may be that the right resolution for this ticket is just that we need a clearer error (or that #21862 is a dupe of #17304, and part of the resolution of #17304 should be a clearer error).

[Carl Meyer]

It looks like with the new app-loading stuff, the correct way to get the base path for an app is now app_config.path. And by default app_config.path now uses module.__path__[0] (which is causing this error) rather than __file__ (which caused the error prompting #17304, pre-app-loading). We _could_ resolve this bug by changing module.__path__[0] to list(module.__path__)[0], and then we would "support" apps that are namespace packages, although if they actually are multi-located namespace packages, we would be arbitrarily choosing one location (the first found on sys.path) and only supporting templates etc in that location and no others.

Alternatively (and perhaps less confusingly), AppConfig could check for len(module.__path__) > 1 and raise a clear error that apps cannot be namespace packages.

Third option, and most invasive, would be to "fully" support apps as namespace packages, by replacing AppConfig.path with AppConfig.paths, and requiring any code using it (e.g. the app-directories template loader) to potentially look in multiple paths for each app.

Personally I think I'd vote for the second option (raise an error); I think this bug was filed simply as a result of someone forgetting the __init__.py, not because there was a real use case for a namespace package app.

At some point in the future when everyone is using Python 3.3+, it's possible that __init__.py becomes a historical relic and everyone just leaves it out as a matter of course, even when they don't really need a namespace package. If that happens, we would need to revisit this choice.

(As a side note, it appears AppStaticStorage is still using app_module.__file__ and should perhaps be updated to use app_config.path.)

[Carl Meyer]

Replying to bmispelon:

Since apps without an __init__.py file worked with Django 1.6, I'm going to mark this ticket as a release blocker as well.

In my testing, this is not true. With Django 1.6.1 I get the same error reported in #17304.

Based on that, I am going to close this as a duplicate of #17304. The details have changed due to app-loading, but the core issue, and options for resolution, remain the same.

[elbaschid]

Thanks for looking into this. I appreciate everyone's time. I wasn't aware of the fact that namespace packages aren't support and will be putting the __init__.py back into my apps.

[Aymeric Augustin]

I confirm Carl's analysis, especially regarding app-loading.