Opened 9 years ago

Closed 9 years ago

#25246 closed Bug (fixed)

Python 3 error in runserver when a directory is missing __init__.py

Reported by: Jessamyn Smith Owned by: Caio Ariede
Component: Core (Other) Version: dev
Severity: Normal Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Carl Meyer)

Using Python 3.4.2
One app directory is missing a number of files, including init.py
This results in an error when trying to run any manage.py command.
It seems odd that the app directory is showing up multiple times in the path, and that duplication seems to cause issues for Django.

Here is an example traceback:

(testenv)~/Development/testenv-dev/testenv$ python manage.py runserver
Traceback (most recent call last):
  File "/Users/jessamyn/.virtualenvs/testenv/lib/python3.4/site-packages/django/apps/config.py", line 100, in create
    entry = module.default_app_config
AttributeError: 'module' object has no attribute 'default_app_config'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/jessamyn/.virtualenvs/testenv/lib/python3.4/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/Users/jessamyn/.virtualenvs/testenv/lib/python3.4/site-packages/django/core/management/__init__.py", line 354, in execute
    django.setup()
  File "/Users/jessamyn/.virtualenvs/testenv/lib/python3.4/site-packages/django/__init__.py", line 21, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/Users/jessamyn/.virtualenvs/testenv/lib/python3.4/site-packages/django/apps/registry.py", line 85, in populate
    app_config = AppConfig.create(entry)
  File "/Users/jessamyn/.virtualenvs/testenv/lib/python3.4/site-packages/django/apps/config.py", line 103, in create
    return cls(entry, module)
  File "/Users/jessamyn/.virtualenvs/testenv/lib/python3.4/site-packages/django/apps/config.py", line 41, in __init__
    self.path = self._path_from_module(app_module)
  File "/Users/jessamyn/.virtualenvs/testenv/lib/python3.4/site-packages/django/apps/config.py", line 70, in _path_from_module
    "with a 'path' class attribute." % (module, paths))
django.core.exceptions.ImproperlyConfigured: The app module <module 'emptyapp' (namespace)> has multiple filesystem locations (['/Users/jessamyn/Development/testenv-dev/testenv/emptyapp', '/Users/jessamyn/Development/testenv-dev/testenv/emptyapp', '/Users/jessamyn/Development/testenv-dev/testenv/emptyapp']); you must configure this app with an AppConfig subclass with a 'path' class attribute.

Change History (26)

comment:1 by Carl Meyer, 9 years ago

Component: UncategorizedCore (Other)
Description: modified (diff)
Triage Stage: UnreviewedAccepted

comment:2 by Carl Meyer, 9 years ago

I don't know how or why a Python 3 namespace package would end up with a __path__ containing the same directory multiple times (seems like a bug in Python, if someone is able to track down the cause), but regardless Django may as well be robust against it. We should just de-dupe __path__, probably by converting to a set, in AppConfig._path_to_module.

Version 0, edited 9 years ago by Carl Meyer (next)

comment:3 by Caio Ariede, 9 years ago

Owner: changed from nobody to Caio Ariede
Status: newassigned

in reply to:  description ; comment:4 by Caio Ariede, 9 years ago

I couldn't reproduce this issue.

jessamynsmith - could you please provide more information on how to reproduce it?

in reply to:  4 comment:5 by Jessamyn Smith, 9 years ago

Replying to caioariede:

I couldn't reproduce this issue.

jessamynsmith - could you please provide more information on how to reproduce it?

Here is a repository which replicates the issue:

https://github.com/jessamynsmith/django-error-python3

comment:6 by Caio Ariede, 9 years ago

This is odd.

Seems to be related to this bug in Python: http://bugs.python.org/issue19469

Could you please give the output for the following commands?

python --version
python -c 'import no_init; print(no_init.__path__)'

Thanks!

comment:7 by Jessamyn Smith, 9 years ago

$ python --version
Python 3.4.2
$ python -c 'import no_init; print(no_init.path)'
_NamespacePath(['/Users/jessamyn/Development/django_error_py3/django_error_py3/no_init', '/Users/jessamyn/Development/django_error_py3/django_error_py3/no_init'])

comment:8 by Carl Meyer, 9 years ago

@caioariede I don't think we need or want an end-to-end test for this that tries to recreate the actual Python bug using a real package (it seems it's platform-specific). It would be sufficient to just add a unit test using a fake module object with a __path__ attribute containing dupes, and add the de-duping logic in _path_from_module as mentioned above, and reference the Python bug url in a comment.

in reply to:  8 comment:9 by Caio Ariede, 9 years ago

Replying to carljm:

@caioariede I don't think we need or want an end-to-end test for this that tries to recreate the actual Python bug using a real package (it seems it's platform-specific). It would be sufficient to just add a unit test using a fake module object with a __path__ attribute containing dupes, and add the de-duping logic in _path_from_module as mentioned above, and reference the Python bug url in a comment.

Yes, I will do. Seems to be related to Python 3.4.2 (I want to check), I couldn't reproduce it with 3.4.3.

comment:10 by Tim Graham, 9 years ago

If it's fixed in 3.4.3, then I suggest to close as "won't fix" as we only officially support the latest micro release of each Python series.

comment:11 by Carl Meyer, 9 years ago

Yes, I agree with Tim -- if it's fixed in 3.4.3 then we don't need to do anything.

comment:12 by Caio Ariede, 9 years ago

Has patch: set
Version: 1.7master

Cannot reproduce it with Python 3.4.2 too. I'm assuming this bug is still alive in Python 3.x

Pull request: https://github.com/django/django/pull/5182

comment:13 by Tim Graham, 9 years ago

Jessamyn, it looks like you are on Mac. How did you install Python on your machine? I also haven't been able to reproduce the problem on Python 3.4.0 and Ubuntu 14.04. Can you test the proposed patch?

in reply to:  13 comment:14 by Jessamyn Smith, 9 years ago

Replying to timgraham:

Jessamyn, it looks like you are on Mac. How did you install Python on your machine? I also haven't been able to reproduce the problem on Python 3.4.0 and Ubuntu 14.04. Can you test the proposed patch?

Sorry, I tried to reply to this previously and it looks as if my reply was lost. I installed Python using homebrew. I have not had a chance to try to get 3.4.0. I had the issue on 3.4.2 and not on 3.4.3. I will try to get a chance to test the patch later today.

comment:15 by Jessamyn Smith, 9 years ago

I can confirm that the proposed patch fixes the issue.

comment:16 by Tim Graham, 9 years ago

Are you saying that the issue is fixed on Python 3.4.3 or that you didn't have a chance to test there?

in reply to:  16 comment:17 by Jessamyn Smith, 9 years ago

Replying to timgraham:

Are you saying that the issue is fixed on Python 3.4.3 or that you didn't have a chance to test there?

I tested using python 3.4.3

comment:18 by Caio Ariede, 9 years ago

I'm on Mac and I couldn't reproduce this issue with 3.4.2 or 3.4.3.

The Python bug report says it even affects Python 3.5.

comment:19 by Tim Graham, 9 years ago

Jessamyn, it'd be great if you could provide a test case for the Python bug or at least nail down the reason why you can reproduce the problem but no one else who has tried can. I'm a bit reluctant to commit the patch until we figure that out.

in reply to:  19 comment:20 by Jessamyn Smith, 9 years ago

Replying to timgraham:

Jessamyn, it'd be great if you could provide a test case for the Python bug or at least nail down the reason why you can reproduce the problem but no one else who has tried can. I'm a bit reluctant to commit the patch until we figure that out.

So manage.py in that repo runs fine for the rest of you, using python 3.x and django 1.8 on OS X?

comment:21 by Caio Ariede, 9 years ago

Yes, it runs fine for me. Using either 3.4.2 and 3.4.3 on OSX.

comment:22 by Carl Meyer, 9 years ago

Personally I think we should just commit the patch. It's harmless, correct, improves robustness, and it has zero downside.

comment:23 by Tim Graham, 9 years ago

I'm not entirely opposed, but I think it would nice not to commit a patch that references a Python bug which is closed as "cannot reproduce" which seems to be where that bug is headed unless it gets some help. Of course, if we cannot understand the issue any better, we can just remove the ticket reference in our patch.

in reply to:  23 comment:24 by Jessamyn Smith, 9 years ago

Replying to timgraham:

I'm not entirely opposed, but I think it would nice not to commit a patch that references a Python bug which is closed as "cannot reproduce" which seems to be where that bug is headed unless it gets some help. Of course, if we cannot understand the issue any better, we can just remove the ticket reference in our patch.

I'm not sure what else to investigate. My system:
OS X 10.10.5
Using Terminal.app with bash
Python 3.4.3 installed using homebrew
virtualenv==13.0.3
virtualenvwrapper==4.3.3.dev34

I just tested and the old code also fails without using virtualenv. I wonder why it doesn't fail on others' machines.

comment:25 by Tim Graham, 9 years ago

Triage Stage: AcceptedReady for checkin

All right, I guess we just merge the patch and move on to more important things.

comment:26 by Tim Graham <timograham@…>, 9 years ago

Resolution: fixed
Status: assignedclosed

In 6a98396:

Fixed #25246 -- Guarded against duplicate paths in AppConfig.

Note: See TracTickets for help on using tickets.
Back to Top