Removing the magic

The "magic-removal" branch aims to make several sweeping changes to the Django codebase, removing warts that Django has accumulated over the years. Most changes involve the database API and removing some of its unneeded magic, and other changes involve improving the framework's simplicity and usability.

These changes will be integrated into the next Django release, 0.92.

This document explains the changes in the branch.

How to get the branch

Play with it! The branch is available via Subversion at ​http://code.djangoproject.com/svn/django/branches/magic-removal .

Database changes you'll need to make

Rename core database tables

We've renamed a bunch of the core Django tables. To upgrade, execute this SQL in your database:

ALTER TABLE auth_groups RENAME TO auth_group;
ALTER TABLE auth_groups_permissions RENAME TO auth_group_permissions;
ALTER TABLE auth_messages RENAME TO auth_message;
ALTER TABLE auth_permissions RENAME TO auth_permission;
ALTER TABLE auth_users RENAME TO auth_user;
ALTER TABLE auth_users_groups RENAME TO auth_user_groups;
ALTER TABLE auth_users_user_permissions RENAME TO auth_user_user_permissions;
ALTER TABLE content_types RENAME TO django_content_type;
ALTER TABLE core_sessions RENAME TO django_session;
ALTER TABLE django_flatpages RENAME TO django_flatpage;
ALTER TABLE django_flatpages_sites RENAME TO django_flatpage_sites;
ALTER TABLE django_redirects RENAME TO django_redirect;
ALTER TABLE packages RENAME TO django_package;
ALTER TABLE sites RENAME TO django_site;

WorldMaker: It seems that I needed to rename the associated id sequences (in Postgres) as well. Is this standard? Should this be added to the above SQL? It should impact PostgreSQL installations the most, and apparently several versions won't properly rename the sequences.

Database table-naming scheme has been changed

Database table names formerly were created by joining the app_label and module_name. Example: polls_polls.

Because there's no longer any pluralization, database table names are now formed by joining the app_label and model name (singular). Example: polls_poll.

As always, this behavior can be overridden on a per-model basis by specifying the db_table attribute in class Meta in your model.

To upgrade, you'll either have to explicitly set db_table in your models or rename your database tables to fit the new naming scheme Django expects.

Code changes you'll need to make

Model class and Field classes renamed/relocated

Change your models to import from django.db.models instead of django.core.meta.

from django.db import models

class Person(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)

Interact directly with model classes, not with magic modules

Import the model class directly from the module in which it was defined. No more django.models.* magic.

from myproject.people.models import Person
p = Person(first_name='John', last_name='Smith')
p.save()

Namespace simplification

See NamespaceSimplification for details.

Changes to model syntax

• class META should now be class Meta. The latter is easier on the eyes.

• The following are no longer valid parameters to class Meta and should be removed:
  • module_name
  • admin (See "Moved admin options to 'class Admin'" below.)
  • exceptions (Just put your exceptions in the module that contains the models and access them normally.)
  • module_constants (Just put your constants in the module that contains the models and access them normally.)
  • where_constraints (Just use a custom manager. See "Custom managers, and multiple managers" below.)

Moved admin options to 'class Admin'

Instead of admin=meta.Admin in the class META, all admin options are in an inner class Admin.

Old:

class Person(meta.Model):
    first_name = meta.CharField(maxlength=30)
    last_name = meta.CharField(maxlength=30)
    class META:
        admin = meta.Admin(
            list_display = ('first_name', 'last_name')
        )

New:

class Person(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)
    class Admin:
        list_display = ('first_name', 'last_name')

Model methods no longer automatically have access to datetime and db modules

Formerly, each model method magically had access to the datetime module and to the variable db, which represents the current database connection. Now, those have to be imported explicitly.

Old:

    def some_method(self):
        print datetime.datetime.now()
        cursor = db.cursor()
        cursor.execute("UPDATE something;")

New:

import datetime
from django.db import connection

# ...

    def some_method(self):
        print datetime.datetime.now()
        cursor = connection.cursor()
        cursor.execute("UPDATE something;")

Access table-level DB API functions via model classes, not with magic modules

All "table-level" functions -- ways of retrieving records tablewide rather than performing instance-specific tasks -- are now accessed via a model class's objects attribute. They aren't direct methods of a model instance object because we want to keep the "table-wide" and "row-specific" namespaces separate.

from myproject.people.models import Person
p_list = Person.objects.get_list()
p = Person.objects.get_object()

This doesn't work from an instance.

p = Person.objects.get_object(pk=1)
p.objects.get_list() # Raises AttributeError

Override default manager name ("objects")

If a model already has an objects attribute, you'll need to specify an alternate name for the magic objects.

class Person(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)
    objects = models.TextField()
    people = models.Manager()

p = Person(first_name='Mary', last_name='Jones', objects='Hello there.')
p.save()
p.objects == 'Hello there.'
Person.people.get_list()

Custom managers, and multiple managers

You can create as many managers as you want. When necessary (such as on the admin), Django will use the first one defined, in order.

If you define at least one custom manager, it will not get the default "objects" manager.

class Person(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)
    people = models.Manager()
    fun_people = SomeOtherManager()

Added a more powerful way of overriding model methods, removed hard-coded _pre_save(), _post_save(), etc.

Proper subclassing of methods now works, so you can subclass the automatic save() and delete() methods. This removes the need for the _pre_save(), _post_save(), _pre_delete() and _post_delete() hooks -- all of which have been removed. Example:

class Person(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)

    def save(self):
        self.do_something()
        super(Person, self).save() # Call the "real" save() method.
        self.do_something_else()

You can even skip saving (as requested in #1014).

class Person(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)

    def save(self):
        if datetime.date.today() > datetime.date(2005, 1, 1):
            super(Person, self).save() # Call the "real" save() method.
        else:
            # Don't save.
            pass

Database connection relocated/renamed

For any code that uses the raw database connection, use django.db.connection instead of django.core.db.db.

Old:

from django.core.db import db
cursor = db.cursor()

New:

from django.db import connection
cursor = connection.cursor()

Backend-specific functions, if you should need them, are available at django.db.backend.

Old:

from django.core import db
db.quote_name('foo')

New:

from django.db import backend
backend.quote_name('foo')

Also, the various backend functionality has been split into three separate modules for each backend -- base.py, creation.py and introspection.py. This is purely for performance and memory savings, so that basic, everyday Django usage doesn't have to load the introspective functionality into memory.

Renamed DoesNotExist exception

Instead of people.PersonDoesNotExist, it's Person.DoesNotExist.

Old:

from django.models.myapp import people
try:
    people.get_object(pk=1)
except people.PersonDoesNotExist:
    print "Not there"

New:

from path.to.myapp.models import Person
try:
    Person.objects.get_object(pk=1)
except Person.DoesNotExist:
    print "Not there"

Moved admin URLconf to shorten its path

You'll need to change your URLconf to include the new location.

• Old: django.contrib.admin.urls.admin
• New: django.contrib.admin.urls

get_object_or_404 and get_list_or_404 now take model classes, not modules

Old:

get_object_or_404(polls, pk=1)

New:

get_object_or_404(Poll, pk=1)

Moved "auth" and "core" models to django.contrib

See ​http://groups.google.com/group/django-developers/browse_thread/thread/276d071a74543448/7d4b1c40c2d53393

• Old: django.models.auth
• New: django.contrib.auth.models

• Old: django.models.core.sites
• New: django.contrib.sites.models

• Old: django.models.core.contenttypes
• New: django.contrib.contenttypes.models

• Old: django.models.core.packages
• New: django.contrib.contenttypes.models ("Packages" will most likely be removed in the future.)

Moved Session model and middleware from core to django.contrib

The location of the session middleware has changed.

• Old: django.middleware.sessions.SessionMiddleware
• New: django.contrib.sessions.middleware.SessionMiddleware

Make sure to update your MIDDLEWARE_CLASSES setting, if you're using sessions.

Also, the Session model has moved from django/models/core.py to django/contrib/sessions/models.py. If you're accessing the Session model for some reason, note that location change.

Changed the parameters you pass to generic views

Because there's no longer a concept of module_name, the "info_dicts" passed to ​generic views no longer accept "app_label" and "module_name". Instead, pass the parameter "model", which should be your model class.

These examples assume models live in myproject/blog/models.py.

Old:

info_dict = {
    'app_label': 'blog',
    'module_name': 'entries'
}

New:

from myproject.blog.models import Entry
info_dict = {
    'model': Entry
}

Changed template names in generic views

Because there's no longer a concept of module_name, ​generic views no longer create templates based on the module_name. Wherever they used module_name, they now use model_name, a lowercase version of the model name.

Note that app_label remains the same.

These examples assume models live in myproject/blog/models.py.

• Old: blog/entries_archive.html
• New: blog/entry_archive.html

Moved settings into an instance

To make it easier to switch settings in situations where you would need multiple different settings - for example when trying to use multiple django projects within one server context or when using Django apps within a bigger WSGI scenario - the settings were moved out of a dedicated module django.conf.settings into an instance in the django.conf module. So now you need to import the settings object and reference settings as attributes of that instance.

Wrappers around the Django machinery can make use of this by exchanging the settings instance with a proxy instance that delegates attribute access to a per-thread or per-location global.

• Old: from django.conf.settings import LANGUAGE_CODE
• New: from django.conf import settings

Removed SilentVariableFailure exception

Old behavior: Any exception that subclasses django.core.template.SilentVariableFailure fails silently in the template system.

New behavior: Any exception that has a silent_variable_failure attribute fails silently in the template system. django.core.template.SilentVariableFailure no longer exists.

New functionality you can start using

Models support properties

Unlike before, properties are supported on models.

from django.db import models

class Person(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)

    def _get_full_name(self):
        return "%s %s" % (self.first_name, self.last_name)
    full_name = property(_get_full_name)

You can override table-level functions

You can override any table-level functions, such as get_list() or get_object(). Do this by creating a custom models.Manager subclass and passing it to your model.

from django.db import models
class PersonManager(models.Manager):
    def get_list(self, **kwargs):
        # Changes get_list() to hard-code a limit=10.
        kwargs['limit'] = 10
        return models.Manager.get_list(self, **kwargs) # Call the "real" get_list() method.

class Person(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)
    objects = PersonManager()

If a manager needs to access its associated class, it should use self.klass. Example:

class PersonManager(models.Manager):
    def get_fun_person(self):
        try:
            return self.get_object(fun__exact=True)
        except self.klass.DoesNotExist:
            print "Doesn't exist."

Stuff that still needs to be done

Automatic manipulators

Status: Mostly done, with some quirks left

Old:

from django.models.myapp import people
m1 = people.AddManipulator()
m2 = people.ChangeManipulator(3)

New:

from path.to.myapp.models import Person
m1 = Person.AddManipulator()
m2 = Person.ChangeManipulator(3)

Change subclassing syntax

Status: Not done yet

Database lookup API changes

Status: Not done yet

See DescriptorFields, rjwittams' proposal on the API changes.