ImportError when trying to use models from standalone script.

[eric@…]

This is a regression between 1.2-beta-1 and 1.2-rc-1

I have a maintenance script that accesses the database via the same django models I'm using for the website. When I upgraded from 1.2-beta-1 to 1.2-rc-1, this script started failing with the following traceback unless the PWD is the django project directory:

Traceback (most recent call last):
  File "test.py", line 19, in <module>
    exam = Exam.objects.get(id=id)
  File "/Users/kd5bjo/sentinel/packages/Django-1.2-rc-1/django/db/models/manager.py", line 132, in get
    return self.get_query_set().get(*args, **kwargs)
  File "/Users/kd5bjo/sentinel/packages/Django-1.2-rc-1/django/db/models/query.py", line 333, in get
    clone = self.filter(*args, **kwargs)
  File "/Users/kd5bjo/sentinel/packages/Django-1.2-rc-1/django/db/models/query.py", line 550, in filter
    return self._filter_or_exclude(False, *args, **kwargs)
  File "/Users/kd5bjo/sentinel/packages/Django-1.2-rc-1/django/db/models/query.py", line 568, in _filter_or_exclude
    clone.query.add_q(Q(*args, **kwargs))
  File "/Users/kd5bjo/sentinel/packages/Django-1.2-rc-1/django/db/models/sql/query.py", line 1131, in add_q
    can_reuse=used_aliases)
  File "/Users/kd5bjo/sentinel/packages/Django-1.2-rc-1/django/db/models/sql/query.py", line 1026, in add_filter
    negate=negate, process_extras=process_extras)
  File "/Users/kd5bjo/sentinel/packages/Django-1.2-rc-1/django/db/models/sql/query.py", line 1182, in setup_joins
    field, model, direct, m2m = opts.get_field_by_name(name)
  File "/Users/kd5bjo/sentinel/packages/Django-1.2-rc-1/django/db/models/options.py", line 291, in get_field_by_name
    cache = self.init_name_map()
  File "/Users/kd5bjo/sentinel/packages/Django-1.2-rc-1/django/db/models/options.py", line 321, in init_name_map
    for f, model in self.get_all_related_m2m_objects_with_model():
  File "/Users/kd5bjo/sentinel/packages/Django-1.2-rc-1/django/db/models/options.py", line 396, in get_all_related_m2m_objects_with_model
    cache = self._fill_related_many_to_many_cache()
  File "/Users/kd5bjo/sentinel/packages/Django-1.2-rc-1/django/db/models/options.py", line 410, in _fill_related_many_to_many_cache
    for klass in get_models():
  File "/Users/kd5bjo/sentinel/packages/Django-1.2-rc-1/django/db/models/loading.py", line 167, in get_models
    self._populate()
  File "/Users/kd5bjo/sentinel/packages/Django-1.2-rc-1/django/db/models/loading.py", line 61, in _populate
    self.load_app(app_name, True)
  File "/Users/kd5bjo/sentinel/packages/Django-1.2-rc-1/django/db/models/loading.py", line 76, in load_app
    app_module = import_module(app_name)
  File "/Users/kd5bjo/sentinel/packages/Django-1.2-rc-1/django/utils/importlib.py", line 35, in import_module
    __import__(name)
ImportError: No module named app

The script is being executed with the following environment variables:

PYTHONPATH=.:./packages/Django-1.2-rc-1:./packages/south:.:/Users/kd5bjo/Library/Python/2.5/site-packages:/Library/Python/2.5/site-packages:/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5
DJANGO_SETTINGS_MODULE=checkpoint.settings
PWD=/Users/kd5bjo/sentinel

The django project root is /Users/kd5bjo/sentinel/checkpoint, and the django app in question is called "app"

[Karen Tracey]

This is likely due to r12950, and it appears to me that that fix is revealing an existing problem in the environment setup for the script. It looks to me like /Users/kd5bjo/sentinel/checkpoint should be included in the PYTHONPATH for this script. When you try to execute the Exam.objects.get(id=id) query, Django needs to load all models from all INSTALLED_APPS to ensure all the model relationships are set up properly. It cannot load the models for app because app isn't found on the Python path. Due to the bug fixed by r12950, that legitimate ImportError was being suppressed, when it should not have been: it's a real problem.

Does your script not actually use any of the app models? That may be why you were not seeing any problems with it. Or have you elsewhere caused these models to get loaded by importing checkpoint.app.models? (I'm not entirely sure if that would work to get things set up properly when you have just app listed in INSTALLED_APPS.)

If you've got a script running using settings that refer to an app, that app needs to be findable via the Python path for things to work properly. The fix here should be to properly set the PYTHONPATH for this script.

[eric@…]

Minimal test case

[eric@…]

My script does import and use checkpoint.app.models (which Exam is one of). I would rather not pollute the module namespace of my script with that of my django project if possible. I have attached a reproducible test case.

From the django-bug-report directory, "./runpy.sh fail.py" will throw the error if run against 1.2-rc-1, but succeed when run against 1.2-beta-1

Historically, Django hasn't required the project root to be in the Python path to use the models, so long as you can import them.

[Russell Keith-Magee]

Karen is correct - the old implementation was hiding an import error in your code. Your settings defines INSTALLED_APPS = ( ..., 'app'), but that isn't a module description that can be imported. The old implementation was hiding this flaw; the new implementation doesn't.

This isn't a case of the new error checking being too harsh, either. Put the following code into fail.py, and run it with beta-1:

from django.db.models import get_app
print get_app('app')

You should get the error "django.core.exceptions.ImproperlyConfigured: App with label app could not be found'

This tells us is that the import process isn't working - even though you have 'app' in your INSTALLED_APPS, and you're importing bug.app.models, the 'app' application isn't being successfully registered with Django's model registry. This was being masked by the old import scheme. Although you may not have been observing any problems, there would be many introspective operations in Django that wouldn't work as expected.

There are two possible fixes for your code:

1. As you suggest, put the project directory in the project path, so that 'app' is a valid path description
2. Modify your INSTALLED_APPS description to use a valid import path - i.e., specify 'bug.app', not 'app'.

Closing invalid; the r12950 implementation is correct.

[Jacob]

Milestone 1.2 deleted