Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#18179 closed Bug (invalid)

Management can't load custom commands when separately packaged apps share a common base module

Reported by: nOw2 Owned by: nobody
Component: Core (Management commands) Version: 1.4
Severity: Normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by ramiro) loads custom commands for by finding the path of the module and examining file directly.

This fails when apps are within packages that share a common base name, but where the files are NOT in the same directories, example:

  • app 1: company.division.project_a.app1 stored in path packages/company.subdivision.project_a.app1
  • app 2: company.division.project_b.app2 stored in path packages/company.subdivision.project_b.app2

Custom commands in app 2 will not be found.

A code trace from pdb follows. Excuse the rather complicated example, but this is from a genuine problem and only the modules names have been changed.

> site-packages/django/core/management/
-> parts.append('management')
(Pdb) parts
['company', 'subdivision', 'project_b', 'app2']

. . .

> site-packages/django/core/management/
-> f, path, descr = imp.find_module(part, path and [path] or None)
ImportError: 'No module named project_b'
> site-packages/django/core/management/
-> f, path, descr = imp.find_module(part, path and [path] or None)
(Pdb) l
 57  	        if os.path.basename(os.getcwd()) != part:
 58  	            raise e
 60  	    while parts:
 61  	        part = parts.pop()
 62  ->	        f, path, descr = imp.find_module(part, path and [path] or None)
 63  	    return path
 65  	def load_command_class(app_name, name):
 66  	    """
 67  	    Given a command name and an application name, returns the Command
(Pdb) part
(Pdb) path

Issue was found in 1.3.1 but the code appears the same in the current trunk version:

Change History (5)

comment:1 Changed 3 years ago by anonymous

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Type changed from Uncategorized to Bug

comment:2 Changed 3 years ago by nOw2

The following replacement function fixes the issue, but introduces a performance problem - there's a small but noticeable delay on starting when the project contains many applications (testing with ~30 here).

def find_management_module(app_name):
    Determines the path to the management module for the given app_name,
    without actually importing the application or the management module.

    Raises ImportError if the management module cannot be found for any reason.

    path = os.path.join(__import__(app_name, fromlist="mangement").__path__[0], "management")
    if not os.path.exists(path):
        raise ImportError("No management commands for %s" % app_name)
    return path

comment:3 Changed 3 years ago by andrewgodwin

  • Resolution set to invalid
  • Status changed from new to closed

It looks like you've got packages with "." in their directory names - that's not a valid Python module name, and while "import" may import it, it does a lot of things it shouldn't do. Module names (and thus directory names) should consist only of letters, numbers and the underscore character.

comment:4 Changed 3 years ago by nOw2

The packages don't have a period in their names, but the directory above the python path does though I don't believe that has an impact.

i.e. for:


Python searches this: "company/subdivision/project_a/app1/"
Package name is "company.subdivision.project_a.app1"

The issue comes from developing each package separately but in the same Python namespace, which is the method we've come up with to handle this particular large project.

The issue comes in because there are many such packages like this:


The way the current find_management_module() works is to find the first file/directory on disk that corresponds to the Python module "company", even though there may be many such directories. i.e. the package name "company.subdivision.project_a.app1" has a unique, but the package name "company" is not.

(nb, the _ _ init _ in the "shared" name components looks like this:

    import pkg_resources



In the above example, "import company.subdivision.project_b.app1" causes find_management_module() to fail when it can't find project_b in the directory "packages/company.subdivision.project_a.app1/company/subdivision/". That directory is first on the path to match company.subdivision but is not the correct one for company.subdivision.project_b.app1!

I recognise that this is something of an edge case.
I have yet to test if the issue continues when the packages are installed via setuptools/distribute, I'm unsure if the issue would persist once installed to site-packages or into eggs. My workaround patch is in our current production release.

Version 0, edited 3 years ago by nOw2 (next)

comment:5 Changed 3 years ago by ramiro

  • Description modified (diff)
Note: See TracTickets for help on using tickets.
Back to Top