= Removing the magic =

The "magic-removal" branch aims to make several sweeping changes to the Django codebase, removing warts that Django has accumulated over the years. Most changes involve the database API and removing some of its unneeded magic, and other changes involve improving the framework's simplicity and usability.

These changes will be integrated into the next Django release, 0.92.

This document explains the changes in the branch.

== How to get the branch ==

Play with it! The branch is available via Subversion at http://code.djangoproject.com/svn/django/branches/magic-removal .

== Overview ==

The biggest changes in magic-removal are:

 * The magic package {{{django.models}}} no longer exists. To use models, just import the model class from wherever it lives on the Python path. Similarly, the magic modules (such as {{{django.models.polls}}} in the tutorial) no longer exist; now, you interact directly with the model class.
 * All automatic pluralization is gone.
 * The lookup API is changing so that many methods will become attributes. For example, {{{choice.get_poll()}}} is now {{{choice.poll}}}.
 * Various packages, such as the Django template system (previously in {{{django.core.template}}}), have been moved around to make importing less verbose and easier to remember.

== Database changes you'll need to make ==

To upgrade from a previous Django installation, you'll need to make some database changes. Obviously, this doesn't apply if you're starting from scratch.

=== Rename core database tables ===

We've renamed a bunch of the core Django tables. To upgrade in MySQL and SQLite, execute this SQL in your database:

{{{
ALTER TABLE auth_groups RENAME TO auth_group;
ALTER TABLE auth_groups_permissions RENAME TO auth_group_permissions;
ALTER TABLE auth_messages RENAME TO auth_message;
ALTER TABLE auth_permissions RENAME TO auth_permission;
ALTER TABLE auth_users RENAME TO auth_user;
ALTER TABLE auth_users_groups RENAME TO auth_user_groups;
ALTER TABLE auth_users_user_permissions RENAME TO auth_user_user_permissions;
ALTER TABLE content_types RENAME TO django_content_type;
ALTER TABLE core_sessions RENAME TO django_session;
ALTER TABLE django_flatpages RENAME TO django_flatpage;
ALTER TABLE django_flatpages_sites RENAME TO django_flatpage_sites;
ALTER TABLE django_redirects RENAME TO django_redirect;
ALTER TABLE packages RENAME TO django_package;
ALTER TABLE sites RENAME TO django_site;
}}}

PostgreSQL is different, because you have to rename sequences, too. To upgrade in PostgreSQL, execute this SQL in your database:

{{{
BEGIN;

ALTER TABLE auth_groups RENAME TO auth_group;
ALTER TABLE auth_groups_id_seq RENAME TO auth_group_id_seq;
ALTER TABLE auth_group ALTER COLUMN id DROP DEFAULT;
ALTER TABLE auth_group ALTER COLUMN id SET DEFAULT nextval('public.auth_group_id_seq'::text);

ALTER TABLE auth_groups_permissions RENAME TO auth_group_permissions;
ALTER TABLE auth_groups_permissions_id_seq RENAME TO auth_group_permissions_id_seq;
ALTER TABLE auth_group_permissions ALTER COLUMN id DROP DEFAULT;
ALTER TABLE auth_group_permissions ALTER COLUMN id SET DEFAULT nextval('public.auth_group_permissions_id_seq'::text);

ALTER TABLE auth_messages RENAME TO auth_message;
ALTER TABLE auth_messages_id_seq RENAME TO auth_message_id_seq;
ALTER TABLE auth_message ALTER COLUMN id DROP DEFAULT;
ALTER TABLE auth_message ALTER COLUMN id SET DEFAULT nextval('public.auth_message_id_seq'::text);

ALTER TABLE auth_permissions RENAME TO auth_permission;
ALTER TABLE auth_permissions_id_seq RENAME TO auth_permission_id_seq;
ALTER TABLE auth_permission ALTER COLUMN id DROP DEFAULT;
ALTER TABLE auth_permission ALTER COLUMN id SET DEFAULT nextval('public.auth_permission_id_seq'::text);

ALTER TABLE auth_users RENAME TO auth_user;
ALTER TABLE auth_users_id_seq RENAME TO auth_user_id_seq;
ALTER TABLE auth_user ALTER COLUMN id DROP DEFAULT;
ALTER TABLE auth_user ALTER COLUMN id SET DEFAULT nextval('public.auth_user_id_seq'::text);

ALTER TABLE auth_users_groups RENAME TO auth_user_groups;
ALTER TABLE auth_users_groups_id_seq RENAME TO auth_user_groups_id_seq;
ALTER TABLE auth_user_groups ALTER COLUMN id DROP DEFAULT;
ALTER TABLE auth_user_groups ALTER COLUMN id SET DEFAULT nextval('public.auth_user_groups_id_seq'::text);

ALTER TABLE auth_users_user_permissions RENAME TO auth_user_user_permissions;
ALTER TABLE auth_users_user_permissions_id_seq RENAME TO auth_user_user_permissions_id_seq;
ALTER TABLE auth_user_user_permissions ALTER COLUMN id DROP DEFAULT;
ALTER TABLE auth_user_user_permissions ALTER COLUMN id SET DEFAULT nextval('public.auth_user_user_permissions_id_seq'::text);

ALTER TABLE content_types RENAME TO django_content_type;
ALTER TABLE content_types_id_seq RENAME TO django_content_type_id_seq;
ALTER TABLE django_content_type ALTER COLUMN id DROP DEFAULT;
ALTER TABLE django_content_type ALTER COLUMN id SET DEFAULT nextval('public.django_content_type_id_seq'::text);

ALTER TABLE core_sessions RENAME TO django_session;

ALTER TABLE django_flatpages RENAME TO django_flatpage;
ALTER TABLE django_flatpages_id_seq RENAME TO django_flatpage_id_seq;
ALTER TABLE django_flatpage ALTER COLUMN id DROP DEFAULT;
ALTER TABLE django_flatpage ALTER COLUMN id SET DEFAULT nextval('public.django_flatpage_id_seq'::text);

ALTER TABLE django_flatpages_sites RENAME TO django_flatpage_sites;
ALTER TABLE django_flatpages_sites_id_seq RENAME TO django_flatpage_sites_id_seq;
ALTER TABLE django_flatpage_sites ALTER COLUMN id DROP DEFAULT;
ALTER TABLE django_flatpage_sites ALTER COLUMN id SET DEFAULT nextval('public.django_flatpage_sites_id_seq'::text);

ALTER TABLE django_redirects RENAME TO django_redirect;
ALTER TABLE django_redirects_id_seq RENAME TO django_redirect_id_seq;
ALTER TABLE django_redirect ALTER COLUMN id DROP DEFAULT;
ALTER TABLE django_redirect ALTER COLUMN id SET DEFAULT nextval('public.django_redirect_id_seq'::text);

ALTER TABLE packages RENAME TO django_package;

ALTER TABLE sites RENAME TO django_site;
ALTER TABLE sites_id_seq RENAME TO django_site_id_seq;
ALTER TABLE django_site ALTER COLUMN id DROP DEFAULT;
ALTER TABLE django_site ALTER COLUMN id SET DEFAULT nextval('public.django_site_id_seq'::text);

COMMIT;
}}}

=== Database table-naming scheme has been changed ===

Database table names formerly were created by joining the {{{app_label}}} and {{{module_name}}}. Example: {{{polls_polls}}}.

Because there's no longer any concept of {{{module_name}}}, database table names are now formed by joining the {{{app_label}}} and model class name (lower case). Example: {{{polls_poll}}}.

As always, this behavior can be overridden on a per-model basis by specifying the {{{db_table}}} attribute in {{{class Meta}}} in your model.

To upgrade, you'll either have to explicitly set {{{db_table}}} in your models or rename your database tables to fit the new naming scheme Django expects. We'd recommend setting {{{db_table}}}, because it's easier.






== Code changes you'll need to make ==

=== Model class and Field classes renamed/relocated ===

Change your models to import from {{{django.db.models}}} instead of {{{django.core.meta}}}.

{{{
#!python
from django.db import models

class Person(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)
}}}

=== Interact directly with model classes, not with magic modules ===

Import the model class directly from the module in which it was defined. No more {{{django.models.*}}} magic.

{{{
#!python
from myproject.people.models import Person
p = Person(first_name='John', last_name='Smith')
p.save()
}}}

=== Namespace simplification ===

See NamespaceSimplification for details.

=== Changes to model syntax ===

 * {{{class META}}} should now be {{{class Meta}}}. The latter is easier on the eyes.

 * The following are no longer valid parameters to {{{class Meta}}} and should be removed:
    * {{{module_name}}}
    * {{{admin}}} (See "Moved admin options to 'class Admin'" below.)
    * {{{exceptions}}} (Just put your exceptions in the module that contains the models and access them normally.)
    * {{{module_constants}}} (Just put your constants in the module that contains the models and access them normally.)
    * {{{where_constraints}}} (Just use a custom manager. See "Custom managers, and multiple managers" below.)

=== Moved admin options to 'class Admin' ===

Instead of {{{admin=meta.Admin}}} in the {{{class META}}}, all admin options are in an inner {{{class Admin}}}.

Old:
{{{
#!python
class Person(meta.Model):
    first_name = meta.CharField(maxlength=30)
    last_name = meta.CharField(maxlength=30)
    class META:
        admin = meta.Admin(
            list_display = ('first_name', 'last_name')
        )
}}}

New:
{{{
#!python
class Person(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)
    class Admin:
        list_display = ('first_name', 'last_name')
}}}

=== Model methods no longer automatically have access to datetime and db modules ===

Formerly, each model method magically had access to the {{{datetime}}} module and to the variable {{{db}}}, which represents the current database connection. Now, those have to be imported explicitly.

Old:
{{{
#!python
    def some_method(self):
        print datetime.datetime.now()
        cursor = db.cursor()
        cursor.execute("UPDATE something;")
}}}

New:
{{{
#!python
import datetime
from django.db import connection

# ...

    def some_method(self):
        print datetime.datetime.now()
        cursor = connection.cursor()
        cursor.execute("UPDATE something;")
}}}

=== Access table-level DB API functions via model classes, not with magic modules ===

All "table-level" functions -- ways of retrieving records tablewide rather than performing instance-specific tasks -- are now accessed via a model class's {{{objects}}} attribute. They aren't direct methods of a model instance object because we want to keep the "table-wide" and "row-specific" namespaces separate.

{{{
#!python
from myproject.people.models import Person
p_list = Person.objects.get_list()
p = Person.objects.get_object()
}}}

This doesn't work from an instance.

{{{
#!python
p = Person.objects.get_object(pk=1)
p.objects.get_list() # Raises AttributeError
}}}

=== Override default manager name ("objects") ===

If a model already has an {{{objects}}} attribute, you'll need to specify an alternate name for the magic {{{objects}}}.

{{{
#!python
class Person(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)
    objects = models.TextField()
    people = models.Manager()

p = Person(first_name='Mary', last_name='Jones', objects='Hello there.')
p.save()
p.objects == 'Hello there.'
Person.people.get_list()
}}}

=== Custom managers, and multiple managers ===

You can create as many managers as you want. When necessary (such as on the admin), Django will use the first one defined, in order.

If you define at least one custom manager, it will not get the default "objects" manager.

{{{
#!python
class Person(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)
    people = models.Manager()
    fun_people = SomeOtherManager()
}}}

=== Added a more powerful way of overriding model methods, removed hard-coded _pre_save(), _post_save(), etc. ===

Proper subclassing of methods now works, so you can subclass the automatic {{{save()}}} and {{{delete()}}} methods. This removes the need for the {{{_pre_save()}}}, {{{_post_save()}}}, {{{_pre_delete()}}} and {{{_post_delete()}}} hooks -- all of which have been removed. Example:

{{{
#!python
class Person(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)

    def save(self):
        self.do_something()
        super(Person, self).save() # Call the "real" save() method.
        self.do_something_else()
}}}

You can even skip saving (as requested in #1014).

{{{
#!python
class Person(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)

    def save(self):
        if datetime.date.today() > datetime.date(2005, 1, 1):
            super(Person, self).save() # Call the "real" save() method.
        else:
            # Don't save.
            pass
}}}

=== Database connection relocated/renamed ===

For any code that uses the raw database connection, use {{{django.db.connection}}} instead of {{{django.core.db.db}}}.

Old:
{{{
#!python
from django.core.db import db
cursor = db.cursor()
}}}

New:
{{{
#!python
from django.db import connection
cursor = connection.cursor()
}}}

Backend-specific functions, if you should need them, are available at {{{django.db.backend}}}.

Old:
{{{
#!python
from django.core import db
db.quote_name('foo')
}}}

New:
{{{
#!python
from django.db import backend
backend.quote_name('foo')
}}}

Also, the various backend functionality has been split into three separate modules for each backend -- {{{base.py}}}, {{{creation.py}}} and {{{introspection.py}}}. This is purely for performance and memory savings, so that basic, everyday Django usage doesn't have to load the introspective functionality into memory.

=== Renamed !DoesNotExist exception ===

Instead of {{{people.PersonDoesNotExist}}}, it's {{{Person.DoesNotExist}}}.

Old:
{{{
#!python
from django.models.myapp import people
try:
    people.get_object(pk=1)
except people.PersonDoesNotExist:
    print "Not there"
}}}

New:
{{{
#!python
from path.to.myapp.models import Person
try:
    Person.objects.get_object(pk=1)
except Person.DoesNotExist:
    print "Not there"
}}}

=== Moved admin URLconf to shorten its path ===

You'll need to change your URLconf to {{{include}}} the new location.

 * Old: {{{django.contrib.admin.urls.admin}}}
 * New: {{{django.contrib.admin.urls}}}

=== get_object_or_404 and get_list_or_404 now take model classes, not modules ===

Old:
{{{
#!python
get_object_or_404(polls, pk=1)
}}}

New:
{{{
#!python
get_object_or_404(Poll, pk=1)
}}}

=== Moved "auth" and "core" models to django.contrib ===

See http://groups.google.com/group/django-developers/browse_thread/thread/276d071a74543448/7d4b1c40c2d53393

 * Old: {{{django.models.auth}}}
 * New: {{{django.contrib.auth.models}}}

 * Old: {{{django.models.core.sites}}}
 * New: {{{django.contrib.sites.models}}}

 * Old: {{{django.models.core.contenttypes}}}
 * New: {{{django.contrib.contenttypes.models}}}

 * Old: {{{django.models.core.packages}}}
 * New: {{{django.contrib.contenttypes.models}}} ("Packages" will most likely be removed in the future.)

=== Moved Session model and middleware from core to django.contrib ===

The location of the session middleware has changed.

 * Old: {{{django.middleware.sessions.SessionMiddleware}}}
 * New: {{{django.contrib.sessions.middleware.SessionMiddleware}}}

Make sure to update your {{{MIDDLEWARE_CLASSES}}} setting, if you're using sessions.

Also, the {{{Session}}} model has moved from django/models/core.py to django/contrib/sessions/models.py. If you're accessing the {{{Session}}} model for some reason, note that location change.

=== Changed the parameters you pass to generic views ===

Because there's no longer a concept of {{{module_name}}}, the "info_dicts" passed to [http://www.djangoproject.com/documentation/generic_views/ generic views] no longer accept {{{"app_label"}}} and {{{"module_name"}}}. Instead, pass the parameter {{{"model"}}}, which should be your model class.

These examples assume models live in {{{myproject/blog/models.py}}}.

Old:
{{{
#!python
info_dict = {
    'app_label': 'blog',
    'module_name': 'entries'
}
}}}

New:
{{{
#!python
from myproject.blog.models import Entry
info_dict = {
    'model': Entry
}
}}}

=== Changed template names in generic views ===

Because there's no longer a concept of {{{module_name}}}, [http://www.djangoproject.com/documentation/generic_views/ generic views] no longer create templates based on the {{{module_name}}}. Wherever they used {{{module_name}}}, they now use {{{model_name}}}, a lowercase version of the model name.

Note that {{{app_label}}} remains the same.

These examples assume models live in {{{myproject/blog/models.py}}}.

 * Old: {{{blog/entries_archive.html}}}
 * New: {{{blog/entry_archive.html}}}

=== Moved settings into an instance ===

To make it easier to switch settings in situations where you would need multiple different settings - for example when trying to use multiple django projects within one server context or when using Django apps within a bigger WSGI scenario - the settings were moved out of a dedicated module {{{django.conf.settings}}} into an instance in the {{{django.conf}}} module. So now you need to import the {{{settings}}} object and reference settings as attributes of that instance.

Wrappers around the Django machinery can make use of this by exchanging the settings instance with a proxy instance that delegates attribute access to a per-thread or per-location global.

 * Old: {{{from django.conf.settings import LANGUAGE_CODE}}}
 * New: {{{from django.conf import settings}}}

=== Removed !SilentVariableFailure exception ===

Old behavior: Any exception that subclasses {{{django.core.template.SilentVariableFailure}}} fails silently in the template system.

New behavior: Any exception that has a {{{silent_variable_failure}}} attribute fails silently in the template system. {{{django.core.template.SilentVariableFailure}}} no longer exists.





== New functionality you can start using ==

=== Models support properties ===

Unlike before, properties are supported on models.

{{{
#!python
from django.db import models

class Person(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)

    def _get_full_name(self):
        return "%s %s" % (self.first_name, self.last_name)
    full_name = property(_get_full_name)
}}}

=== You can override table-level functions ===

You can override any table-level functions, such as {{{get_list()}}} or {{{get_object()}}}. Do this by creating a custom {{{models.Manager}}} subclass and passing it to your model.

{{{
#!python
from django.db import models
class PersonManager(models.Manager):
    def get_list(self, **kwargs):
        # Changes get_list() to hard-code a limit=10.
        kwargs['limit'] = 10
        return models.Manager.get_list(self, **kwargs) # Call the "real" get_list() method.

class Person(models.Model):
    first_name = models.CharField(maxlength=30)
    last_name = models.CharField(maxlength=30)
    objects = PersonManager()
}}}

If a manager needs to access its associated class, it should use {{{self.model}}}. Example:

{{{
#!python
class PersonManager(models.Manager):
    def get_fun_person(self):
        try:
            return self.get_object(fun__exact=True)
        except self.model.DoesNotExist:
            print "Doesn't exist."
}}}






== Stuff that still needs to be done ==

=== Automatic manipulators ===

'''Status: Mostly done, with some quirks left'''

Old:
{{{
#!python
from django.models.myapp import people
m1 = people.AddManipulator()
m2 = people.ChangeManipulator(3)
}}}

New:
{{{
#!python
from path.to.myapp.models import Person
m1 = Person.AddManipulator()
m2 = Person.ChangeManipulator(3)
}}}

=== Change subclassing syntax ===

'''Status: Not done yet'''

See ModelInheritance

=== Database lookup API changes ===

'''Status: Not done yet'''

See DescriptorFields, rjwittams' proposal on the API changes.