#18179 closed Bug (invalid)
Management can't load custom commands when separately packaged apps share a common base module
Reported by: | nOw2 | Owned by: | nobody |
---|---|---|---|
Component: | Core (Management commands) | Version: | 1.4 |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Unreviewed | |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description (last modified by )
django.core.management.find_management_module() loads custom commands for manage.py by finding the path of the module and examining file directly.
This fails when apps are within packages that share a common base name, but where the files are NOT in the same directories, example:
- app 1: company.division.project_a.app1 stored in path
packages/company.subdivision.project_a.app1
- app 2: company.division.project_b.app2 stored in path
packages/company.subdivision.project_b.app2
Custom commands in app 2 will not be found.
A code trace from pdb follows. Excuse the rather complicated example, but this is from a genuine problem and only the modules names have been changed.
> site-packages/django/core/management/__init__.py(43)find_management_module() -> parts.append('management') (Pdb) parts ['company', 'subdivision', 'project_b', 'app2'] . . . > site-packages/django/core/management/__init__.py(62)find_management_module() -> f, path, descr = imp.find_module(part, path and [path] or None) (Pdb) ImportError: 'No module named project_b' > site-packages/django/core/management/__init__.py(62)find_management_module() -> f, path, descr = imp.find_module(part, path and [path] or None) (Pdb) l 57 if os.path.basename(os.getcwd()) != part: 58 raise e 59 60 while parts: 61 part = parts.pop() 62 -> f, path, descr = imp.find_module(part, path and [path] or None) 63 return path 64 65 def load_command_class(app_name, name): 66 """ 67 Given a command name and an application name, returns the Command (Pdb) part 'project_b' (Pdb) path 'packages/company.subdivision.project_a.app1/company/subdivision'
Issue was found in 1.3.1 but the code appears the same in the current trunk version:
https://code.djangoproject.com/browser/django/trunk/django/core/management/__init__.py?rev=17842#L60
Change History (5)
comment:1 by , 13 years ago
Type: | Uncategorized → Bug |
---|
comment:2 by , 13 years ago
comment:3 by , 12 years ago
Resolution: | → invalid |
---|---|
Status: | new → closed |
It looks like you've got packages with "." in their directory names - that's not a valid Python module name, and while "import" may import it, it does a lot of things it shouldn't do. Module names (and thus directory names) should consist only of letters, numbers and the underscore character.
comment:4 by , 12 years ago
The packages don't have a period in their names, but the directory above the python path does though I don't believe that has an impact.
i.e. for:
packages/company.subdivision.project_a.app1/company/subdivision/project_a/app1/
PYTHONPATH="packages/company.subdivision.project_a.app1/"
Python searches this: "company/subdivision/project_a/app1/"
Package name is "company.subdivision.project_a.app1"
The issue comes from developing each package separately but in the same Python namespace, which is the method we've come up with to handle this particular large project.
The issue comes in because there are many such packages like this:
packages/company.subdivision.project_a.app1/company/subdivision/project_a/app1/
packages/company.subdivision.project_a.app2/company/subdivision/project_a/app2/
packages/company.subdivision.project_b.app1/company/subdivision/project_b/app1/
The way the current find_management_module() works is to find the first file/directory on disk that corresponds to the Python module "company", even though there may be many such directories. i.e. the package name "company.subdivision.project_a.app1" has a unique init.py, but the package name "company" is not.
(nb, the _ _ init _ _.py in the "shared" name components looks like this:
import pkg_resources pkg_resources.declare_namespace(__name__)
)
In the above example, "import company.subdivision.project_b.app1" causes find_management_module() to fail when it can't find project_b in the directory "packages/company.subdivision.project_a.app1/company/subdivision/". That directory is first on the path to match company.subdivision but is not the correct one for company.subdivision.project_b.app1!
I recognise that this is something of an edge case.
I have yet to test if the issue continues when the packages are installed via setuptools/distribute, I'm unsure if the issue would persist once installed to site-packages or into eggs. My workaround patch is in our current production release.
comment:5 by , 12 years ago
Description: | modified (diff) |
---|
The following replacement function fixes the issue, but introduces a performance problem - there's a small but noticeable delay on starting manage.py when the project contains many applications (testing with ~30 here).