|Version 2 (modified by 4 years ago) (diff),|
This is an attempt to bring together the current state and long term plans of how application loading is done in Django, what it could do, and why.
Author: Marc Tamlyn. Hence references to "me" or "I"
Application loading refactor
At present application loading is handled in different ways through Django.
Because of some of the limitations of Python, and some peculiarities of how
models work, modules should not be reimported. Consequently Django maintains
a global static, called the application cache. Models are imported from the
application cache transparently when interacting with the database. But
applications listed in
INSTALLED_APPS contain more than just models. Some
contain template tags,
search_indexes.py files from
haystack, template folders and many other features which applications use to
inspect each other, or to contribute objects to a site-wide collection.
For the majority of applications, even reusable applications, anything more than the current system is unnecessary. Models are automatically registered, template tags can be loaded, and templates are available. (Tests are also available but this is a separate issue.)
However, not every application has all of these components - there are many
useful pluggable applications which do not contain models, but must still be
INSTALLED_APPS. A simple example is
which consists of a single template tag. This module must ship with an empty
models.py module to allow it to be registered as an installed app, for that
template tag to be available.
The problem is even more difficult for "meta-applications" - that is to say applications which exist solely to interact with other applications. The canonical example of this is the django admin, but there are many third party applications which do a similar thing. Haystack (the search engine framework) is a good example. Also there are applications which have a plugin archicture, whereby adding a second application to installed apps changes the behaviour of the first application. All of these third party systems have implemented their own ways to do this, with varying degrees of success.
This is an opportunity for Django to do something great for the third party
application community. It will be a very powerful tool, and opens up a range of
magical things which could be done. App loading is magical, but Magic is not
always a bad thing - most Django developers are very pleased they don't have to
individually register every model with Django as they do to register a model
with the admin. You just put it in the
models.py file and it works! This
will allow the magic to be more explicit, and ultimately more consistent
between third party applications.
This is a big change, and almost redefines what an "App" is within Django. To my mind, an app was originally intended to be a fairly self contained part of the system, normally in fact a self contained part of the Web site.
Here is a specification of what an application is, and what it can do. Perhaps not all of these points need to be implemented to be able to integrate this refactor into core, but they should be considered to avoid future backwards incompatibilities.
- An application is a self contained piece of code which contributes
something to the site. It may contibute any number of the following:
- template tags
- static files
- admin configuration
Note Some things are explicitly registered in settings, such as middleware,
database routers and cache backends. These things may be contained within
an application's code base, but they will not be registered automatically
by the presence of that application in the
- An application may contain urls and views. These will generally be explicitly registered, but some applications may choose to attach dynamic urls offer a way of getting the urls from the application rather than directly (similar to how the admin does to it's site class).
- An application may modify another application. That is to say, an application may provide an interface on it's application class for other applications to modify it. This will facilitate a plugin architecture. A classic example of where this will be useful is a Content Management System.
- An application may depend on the presence of another application. It is preferable for the application to depend on a defined contract with another application, rather than explicity depend on that application. This should allow applications to be more loosely coupled, leading potentially to pluggable auth/messages applications. It also allows such applications to be tested perhaps without the presence of the dependant app - a mocked object may suffice.
- An application may inspect other applications to gather information about them. The system should provide tools to easily and safely access this information, whether it is accessing a folder or a python module. Where possible, we should modify parts of Django core & contrib to use this system. Examples would include the app directories template loader, admin site and template tag libraries.
- An exact copy of an application can be deployed within the same project by changing its label. This will have some subtle issues regarding imports.
Implementation plan (incomplete)
Application class. Talk about what APIs would bee needed on that class to achieve the above. Separate by the above points. Ideally, does not need to be contained within the python module it relates to (thus allowing me to subclass an app's Application Class in my own project to tweak how it works, this may or may not be better than passing configuration options)
Application cache. Discuss what hooks this would need to provide, including signals, for the above to work.
Examine any backwards incompatibilities which may result. The app refactor
itself shouldn't introduce any, but the new functionality which it offers and
the resulting changes in other applications such as
contrib.auth may need
This concept has been around in the Django ecosystem for about 4 years now.
The original impetus was to allow custom
attributes on an app (see #3591). The sticking point was often the difficulties of
installing the same app multiple times - there are a bunch of problems with this,
not least including dealing with
from grail.models import Spam. Well, which
An original discussion of the ideas can be found at InstalledAppsRevision.
It was taken on as a Google Summer of Code project in summer 2010 by Arthur
Koziel. Good progress was made, with a refactor of the
AppCache and a new
App class. Some of the design decisions made are discussed here:
The last action on this branch was in summer 2011 by Jannis. Travis Swicegood and Marc Tamlyn have both expressed interest in working further with the project.
Preston has merged master into the existing branch with work done by Jannis and Arthur. (https://github.com/ptone/django/commit/8722bdcc17293baa8b527ae7169b9a465e728d41)
The app cache branch has a good number of tests, but little documentation. We
need to start by working out what has been achieved so far and how it fits in with
the long term plan. The tests for the new classes work, but there are a few other
tests breaking as a result of them expecting certain behaviour about the
Personally, I would like to see code removed from all of these places and the
AppCache made more powerful. These tests will need fixing before any merge
to trunk can be made.