[[PageOutline]]
= Splitting up the settings file =

If you use a source control system (CVS, SVN, ...), or want to publish your application on the web, 
it may be a good idea to move sensitive or machine/user specific settings like database 
passwords and such out of the main settings.py file. 

As discussions on the [http://groups.google.com/group/django-developers django-developers mailing list] have shown everybody has different 
requirements and ideas how to do this. This page is meant to collect some of these ideas for future reference.

One thing to keep in mind is that Django's config files are pure Python. 
This gives you the ultimate flexibility to handle configurations the way you think is best.
Or to quote Adrian Holovaty:

  We don't need a default solution for this. It's not within the scope
  of this project to tell people how they should organize their settings
  files. Take that opportunity to showcase your individualism.

Some of the examples listed below need to be modified for compatibility with Django 1.4 and later. They are marked with "Django <1.4" in the title.


== Links to discussions ==
 * [http://stackoverflow.com/questions/2035733 how to modularize django settings.py?] on stackoverflow.com

== Different settings in different files ==
I believe the first user who came up with this was Hugo, who used this method in the projects he published on [https://simon.bofh.ms/cgi-bin/trac-django-projects.cgi/ his site].

{{{
#!python

SECRET_KEY = open(os.path.expanduser('~/.gallery-secret')).read().strip()
}}}


== ini-style file for deployment ==
This is a solution that Michael Radziej posted to django-developers. His motivation was to be able to store the settings
for an app he published under `/etc` with all the other system config files.

For a similar but more transparent solution see [[SplitSettings#foxsmethod|fox's method]]

`/etc/whatever/settings.ini` (Django <1.4)
{{{
[database]
DATABASE_USER: bla
DATABASE_PASSWORD: XXXXXXXX
DATABASE_HOST: dev
DATABASE_PORT:
DATABASE_ENGINE: mysql
DATABASE_NAME: blo
TESTSUITE_DATABASE_NAME: test_blo

[secrets]
SECRET_KEY: random-string-of-ascii
CSRF_MIDDLEWARE_SECRET: random-string-of-ascii

[cookies]
SESSION_COOKIE_DOMAIN:

# all settings in debug section should be false in productive
environment
# INTERNAL_IPS should be empty in productive environment
[debug]
DEBUG: true
TEMPLATE_DEBUG: true
VIEW_TEST: true
INTERNAL_IPS: 127.0.0.1
SKIP_CSRF_MIDDLEWARE: true

[email]
SERVER_EMAIL: django@localhost
EMAIL_HOST: localhost

# the [error mail] and [404 mail] sections are special. Just add
lines with
#  full name: email_address@domain.xx
# each section must be present but may be empty.
[error mail]
Adam Smith: adam@localhost

[404 mail]
John Wayne: john@localhost
}}}


`/path/to/whatever/settings.py` (Django <1.4)
{{{
#!python

from ConfigParser import RawConfigParser

config = RawConfigParser()
config.read('/etc/whatever/settings.ini')

DATABASE_USER = config.get('database', 'DATABASE_USER')
DATABASE_PASSWORD = config.get('database', 'DATABASE_PASSWORD')
DATABASE_HOST = config.get('database', 'DATABASE_HOST')
DATABASE_PORT = config.get('database', 'DATABASE_PORT')
DATABASE_ENGINE = config.get('database', 'DATABASE_ENGINE')
DATABASE_NAME = config.get('database', 'DATABASE_NAME')
TEST_DATABASE_NAME = config.get('database', 'TESTSUITE_DATABASE_NAME')

SECRET_KEY = config.get('secrets','SECRET_KEY')
CSRF_MIDDLEWARE_SECRET = config.get('secrets', 'CSRF_MIDDLEWARE_SECRET')

SESSION_COOKIE_DOMAIN = config.get('cookies','SESSION_COOKIE_DOMAIN')

DEBUG = config.getboolean('debug','DEBUG')
TEMPLATE_DEBUG = config.getboolean('debug','TEMPLATE_DEBUG')
VIEW_TEST = config.getboolean('debug', 'VIEW_TEST')
INTERNAL_IPS = tuple(config.get('debug', 'INTERNAL_IPS').split())
if config.getboolean('debug', 'SKIP_CSRF_MIDDLEWARE'):
    MIDDLEWARE_CLASSES = tuple([x for x in list(MIDDLEWARE_CLASSES)
                                  if not x.endswith('CsrfMiddleware')])

SERVER_EMAIL = config.get('email', 'SERVER_EMAIL')
EMAIL_HOST = config.get('email', 'EMAIL_HOST')
ADMINS = tuple(config.items('error mail'))
MANAGERS = tuple(config.items('404 mail'))
}}}


== Multiple setting files importing from each other ==
This is my (Steven Armstrong) preferred solution.

Keep application wide, unsensitive settings and sane defaults in your normal `settings.py` file.

`/path/to/whatever/settings.py` (Django <1.4)
{{{
#!python
import os

BASE_DIR = os.path.dirname(os.path.abspath(__file__))

# why is this here?
#PROJECT_DIR = os.path.abspath(os.path.join(BASE_DIR, '..'))


PROJECT_DIR = BASE_DIR

DEBUG = False
TEMPLATE_DEBUG = DEBUG

ADMINS = (
    ('Mr Sysadmin', 'sysadmin@domain.tld'),
)

MANAGERS = ADMINS

DATABASE_ENGINE = 'sqlite3'
DATABASE_NAME = os.path.join(PROJECT_DIR, 'project.db')
DATABASE_USER = ''
DATABASE_PASSWORD = ''
DATABASE_HOST = ''
DATABASE_PORT = ''

TIME_ZONE = 'Europe/Zurich'
LANGUAGE_CODE = 'en-us'
SECRET_KEY = 'secret'

#[more default and app wide settings]

from settings_local import *
}}}

At the end of your normal `settings.py` include * from an other, machine specific config file 
which could look something like this.

`/path/to/whatever/settings_local.py` (Django <1.4)
{{{
#!python

DEBUG = True
TEMPLATE_DEBUG = DEBUG

# don't want emails while developing
ADMINS = ()
MANAGERS = ADMINS

DATABASE_ENGINE = 'mysql'
DATABASE_NAME = 'mydbname'
DATABASE_USER = 'mydbuser'
DATABASE_PASSWORD = 'mydbpassword'
DATABASE_HOST = 'localhost'
DATABASE_PORT = ''

SECRET_KEY = 'random-string-of-ascii'

#[more user/machine specific settings]
}}}

`urls.py`
{{{
import settings
     (r'^static/(?P<path>.*)$', 'django.views.static.serve', {'document_root': settings.BASE_DIR+'/core/static/', 'show_indexes': True}),                      
}}}

`settings.py` goes into CVS, SVN, (put your favorite RCS here)[[BR]]
`settings_local.py` does _not_ go under RCS

If wanted a `settings_local.template` file can be put under version control with
instructions to copy it over to settings_local.py, change it to suite the environment, 
and to never ever commit it to the RCS system.

== Development/Machine Dependant Settings Configuration ==
I personally use this solution.

`/path/to/project/settings.py`
{{{
#!python
# Django project settings loader
import os
ROOT_PATH = os.path.dirname(os.path.abspath(__file__))

# You can key the configurations off of anything - I use project path.
configs = {
    '/path/to/user1/dev/project': 'user1',
    '/path/to/user2/dev/project': 'user2',
    '/path/to/qa/project': 'qa',
    '/path/to/prod/project': 'prod',
}

# Import the configuration settings file - REPLACE projectname with your project
config_module = __import__('config.%s' % configs[ROOT_PATH], globals(), locals(), 'projectname')

# Load the config settings properties into the local scope.
for setting in dir(config_module):
    if setting == setting.upper():
        locals()[setting] = getattr(config_module, setting)
}}}

This settings loader will import the appropriate settings module. I store my settings files in a `config` directory in the root of the project.

`/path/to/project/config/prod.py` (Django <1.4)
{{{
#!python

DEBUG = False
TEMPLATE_DEBUG = DEBUG

# don't want emails while developing
ADMINS = ()
MANAGERS = ADMINS

DATABASE_ENGINE = 'mysql'
DATABASE_NAME = 'mydbname'
DATABASE_USER = 'mydbuser'
DATABASE_PASSWORD = 'mydbpassword'
DATABASE_HOST = 'localhost'
DATABASE_PORT = ''

SECRET_KEY = 'random-string-of-ascii'

# More production settings
}}}

`/path/to/project/config/user1.py` (Django <1.4)
{{{
#!python
# Load the production settings that we will overwrite as needed in our user1 settings file.
from projectname.config.prod import *

DEBUG = True

DATABASE_NAME = 'devdbname'

# More development/maching specific settings
}}}

== Using exec to incorporate local settings ==

I've been using code like this to incorporate local settings:

`settings.py`
{{{
#!python
# Non-machine-specific settings live directly in settings.py
# Machine-specific settings live in another file, perhaps local_settings.conf

exec open('local_settings.conf') in globals()
}}}

This allows the local settings to both read and modify the main settings' values (and it's simple).

As an enhancement, a small amount of privilege separation can be had by making the `local_settings.conf` file unreadable by the uid that runs the web app, opening local_settings.conf in a setuid wrapper, passing its file descriptor to the Django process, and using {{{exec os.fdopen(..., 'r') in globals()}}}. This isn't perfect, since any secrets read from that file are still stored in the Django process, but it does make it harder for someone who has gained access to the user id running the Django app to discover your secrets. (Obviously all of the Python files used by the app must be owned by a user other than the one the app runs as, as well.)

== Using a list of conf files (Transifex) ==

[http://transifex.org/ Transifex] has adopted a solution using an ordered list of multiple `.conf` files inside a settings directory. This helps separating the config options, packaging in Linux distributions, and putting the settings files under `/etc`.

{{{
#!python

$ ls settings/

10-base.conf
20-engines.conf
30-site.conf
...

$ cat settings.py

import os.path
import glob
conffiles = glob.glob(os.path.join(os.path.dirname(__file__), 'settings', '*.conf'))
conffiles.sort()
for f in conffiles:
    execfile(os.path.abspath(f))
}}}

To apply local-only changes, create a `XX-local.conf file`, which can be ignored with your VCS of choice:

{{{
#!python

$ cat settings/51-local.conf

INSTALLED_APPS += ['debug_toolbar']

$ cat .hgignore

syntax: glob
settings/*-local.conf
}}}

== Accessing and modifying existing settings ==

I ([http://djangopeople.net/akaihola Antti Kaihola]) use the following solution for defining local settings. It provides access to the original settings while defining local settings. Note: this only works if you import from the base {{{settings.py}}} or {{{settings/__init__.py}}}.

{{{settings/__init__.py}}}:
{{{
INSTALLED_APPS = ('whatever',)
try:
    from settings.local import *
except ImportError:
    pass
}}}

{{{settings/local.py}}}:
{{{
import sys
globals().update(vars(sys.modules['settings']))

INSTALLED_APPS += ('another_app',)
}}}

Older, clumsier methods I've used:

{{{settings.py}}}:
{{{
INSTALLED_APPS = ('whatever',)
import more_settings
more_settings.modify(globals())
}}}

{{{more_settings.py}}}:
{{{
def modify(settings):
    settings['INSTALLED_APPS'] += ('another_app',)
}}}

alternate more_settings.py with more magic:
{{{
def config(INSTALLED_APPS=(), **other_settings):
    INSTALLED_APPS += ('another_app',)
    del other_settings
    return locals()

def modify(settings):
    settings.update(config(**settings))
}}}

== Settings inheritance ==

I ([http://djangopeople.net/cordis  Sergey Yushckeyev]) have extended proposed solution:

{{{env}}}
{{{
export APP_SETTINGS = 'development.cordis'
}}}

{{{settings/__init__.py}}}
{{{
#!python

CONFIGURATION = 'common'

def deep_update(from_dict, to_dict):
    for (key, value) in from_dict.iteritems():
        if key in to_dict.keys() and isinstance(to_dict[key], dict) and isinstance(value, dict):
            deep_update(value, to_dict[key])
        else:
            to_dict[key] = value

try:
    modules = os.environ.get('APP_SETTINGS', '').split('.')
    current = __name__
    for name in modules:
        module  = getattr(__import__(current, globals(), locals(), [name]), name)
        current += '.' + name
        current_settings = {}
        for setting in dir(module):
            if setting == setting.upper():
                current_settings[setting] = getattr(module, setting)
        deep_update(current_settings, locals())
except ImportError, e:
    print 'Unable to import configuration: %s' % e
}}}

{{{settings/development/cordis.py}}}
{{{
#!python

CONFIGURATION = 'cordis'
}}}

{{{app/models.py}}}
{{{
#!python

from settings import CONFIGURATION
print CONFIGURATION # prints 'cordis'
}}}

=== Setting Inheritance with Hierarchy ===

I ([http://twitter.com/#!/karim_a_nassar Karim A. Nassar]) need to be able to inherit from a base settings file, then for each environment provide overrides. In dev env, we need developer specific settings. The following extends [http://djangopeople.net/cordis  Sergey Yushckeyev's] solution above. 

This script merges {{{settings/common.py}}}, {{{settings/$APP_ENV.py}}} and {{{settings/<developer overrides>.py}}}, in that order.

{{{settings/__init__.py}}}
{{{
#!python
import os, pwd

# certain keys we want to merge instead of copy
merge_keys = ('INSTALLED_APPS', 'MIDDLEWARE_CLASSES')

def deep_update(from_dict, to_dict):
    for (key, value) in from_dict.iteritems():
        if key in to_dict.keys() and isinstance(to_dict[key], dict) and isinstance(value, dict):
            deep_update(value, to_dict[key])
        elif key in merge_keys:
            if not key in to_dict:
                to_dict[key] = ()
            to_dict[key] = to_dict[key] + from_dict[key]
        else:
            to_dict[key] = value

# this should be one of prod, qa, staging, dev. Default to dev for safety.
env = os.environ.get('APP_ENV', 'dev')

# try to load user specific settings
uid = pwd.getpwuid(os.getuid())[0]

modules = ('common', env, uid)
current = __name__
for module_name in modules:
    try:
        module = getattr(__import__(current, globals(), locals(), [module_name]), module_name)
    except ImportError, e:
        print 'ERROR: Unable to import %s configuration: %s' % (module_name, e)
        raise
    except AttributeError, e:
        if env == 'dev' and module_name == uid:
            print 'WARNING: Unable to import %s dev configuration: does %s.py exist?' % (module_name, module_name)
        else:
            raise

    # create a local copy of this module's settings
    module_settings = {}
    for setting in dir(module):
        # all django settings are uppercase, so this ensures we
        # are only processing settings from the dir() call
        if setting == setting.upper():
            module_settings[setting] = getattr(module, setting)
    deep_update(module_settings, locals())

#print locals() # for debugging
}}}

Then, move `settings.py` to `settings/common.py`.


The above defaults to {{{dev}}} env. You could pass in anything to {{{manage.py}}} via the {{{APP_ENV}}} environment variable. Common ones would be {{{prod}}}, {{{staging}}}, {{{qa}}}, etc. For each {{{APP_ENV}}}, you create a file {{{$APP_ENV.py}}}. Finally, create {{{$USER.py}}} file with developer specific overrides. Example files (excepting {{{settings/common.py}}}) below..

`settings/dev.py`
{{{
#!python
import os

# cwd is settings. determine project path
cwd = os.path.dirname(os.path.abspath(__file__)) 
project_path = cwd[:-9] # chop off "settings/"

DEBUG = True
TEMPLATE_DEBUG = DEBUG

TEMPLATE_DIRS = (
    # Put strings here, like "/home/html/django_templates" or "C:/www/django/templates".
    # Always use forward slashes, even on Windows.
    # Don't forget to use absolute paths, not relative paths.

    # automatically inject project templates directory into the dev env
    '%s/templates' % ( project_path )
)
}}}

`settings/$USER.py`
{{{
#!python
DATABASES = {
    'default': {
        'USER': 'me',           
        'PASSWORD': 'i<3django',
    }
}

INSTALLED_APPS = (
    'haystack',
)

HAYSTACK_CONNECTIONS = {
    'default': {
        'ENGINE': 'haystack.backends.solr_backend.SolrEngine',
        'URL': 'http://127.0.0.1:8983/solr'
    },
}
}}}

Of course, you may find that you actually want some of the above settings in `common.py`. Configure however you need for your situation.

=== Simple Package Organization for Environments ===

[http://chesmart.in I] have used a Python package organization similar to those above for per-environment settings, but much simplified: leaving inheritance to be handled by plain Python modules, without deep-merging dictionary magic. Just be explicit! It's easier to know what {{{production.py}}} configures at a glance without having to go dig into data structures in other files. Having per-developer settings is asking for issues with inconsistent configurations to crop up in deployment environments.

{{{
#!sh
[myapp]$ tree settings
settings
├── __init__.py
├── defaults.py
├── dev.py
└── production.py
}}}

The initial {{{settings.py}}} was moved to {{{settings/defaults.py}}}. By default, the settings package will use development configuration, so that local dev Just Works as normal:

{{{settings/__init__.py}}}
{{{
#!python
from dev import *
}}}

{{{settings/dev.py}}}
{{{
#!python
# Load defaults in order to then add/override with dev-only settings
from defaults import *

DEBUG = True
# ... etc.
}}}

For production (or staging, etc.), just add an appropriate module and set Django's conventional [https://docs.djangoproject.com/en/dev/topics/settings/ DJANGO_SETTINGS_MODULE environment variable] in your deployment setup to load it:

{{{settings/production.py}}}
{{{
#!python
# Load defaults in order to then add/override with production-only settings
from defaults import *

DEBUG = False
# ... etc.
}}}

Wherever appropriate for your deployment (such as an Apache {{{SetEnv}}} directive, include the equivalent of:

{{{
#!sh
$ export DJANGO_SETTINGS_MODULE=myapp.settings.production
}}}

Using {{{django-admin.py}}} in deployment rather than {{{manage.py}}} should automatically put your app's package on {{{sys.path}}} so that an import path qualified this way will work.

If you must use a {{{settings/local.py}}} for some reason, virtualenvwrapper's {{{postactivate/postdeactivate}}} hooks are a good place to set {{{DJANGO_SETTINGS_MODULE}}} to point to it for local development per-project. It's advisable to eliminate as many potential discrepancies between dev and production configuration as possible, though.

== Yipit's method ==

[http://tech.yipit.com/author/adam-nelson/ Adam Nelson] from [http://yipit.com/ Yipit] blogged about their method of managing Django settings. See [http://tech.yipit.com/2011/11/02/django-settings-what-to-do-about-settings-py/ Extending Django Settings for the Real World]. TODO: summarize the blog post here.

== Rob Golding's method ==
Rob Golding posted on his blog how he solved this issue. It's a bit of a hack really but it works nicely and allows you to override or extend anything in {{{settings.py}}}.

Add this to the bottom of {{{settings.py}}}:
{{{
#!python
try:
    LOCAL_SETTINGS
except NameError:
    try:
        from local_settings import *
    except ImportError:
        pass
}}}

Now create a {{{local_settings.py}}} with the following:
{{{
#!python
LOCAL_SETTINGS = True
from settings import *
}}}

You can now add things to {{{local_settings.py}}} like this:
{{{
#!python
MIDDLEWARE_CLASSES +=
('debug_toolbar.middleware.DebugToolbarMiddleware',)
INTERNAL_IPS = ('127.0.0.1',)
INSTALLED_APPS += ('debug_toolbar',)
}}}

This does however have the drawback of {{{settings.py}}} being parsed twice but that shouldn't cause any sort of mention worthy performance loss.


== fox's method ==
First write a config file with passwords ad secret keys. Save it somewhere.
This is the `external_config.py`
{{{
#!python
from ConfigParser import ConfigParser, NoOptionError

class ExternalConfig(object):
    """
    Wrapper for ConfigParser
    Reads a section from a config file and provides an object with the
    properties and the values defined in the config.
    """
    def __init__(self, conf_file, section):
        """
        Reads the section "section" from the config "conf_file"
        """
        self._conf_file = conf_file
        self._section = section

        self._config = ConfigParser()
        self._config.read(self._conf_file)

        self._index = len(self._config.options(self._section))

    def __getattr__(self, name):
        """
        Read the property from the config
        """
        if name.startswith('_'):
            return self.__dict__[name]
        try:
            value = self._config.get(self._section, name)
            if value.isdigit():
                return int(value)
            elif value in ('True', 'False'):
                return value == 'True'
            else:
                return value
        except NoOptionError:
            raise AttributeError

    def __getitem__(self, name):
        """
        Read the property from the config
        """
        return self.__getattr__(name)

    def __iter__(self):
        """
        __iter__ method
        """
        return self

    def next(self):
        """
        next method
        """
        if self._index == 0:
            raise StopIteration
        self._index -= 1
        return self._config.options(self._section)[self._index]
}}}

This is `production_settings.py`
{{{
#!python
from external_config import ExternalConfig
import sys

_filename = ""  # add your config file
_section = "mysection"  # add your config section
_this_module = sys.modules[__name__]
_config = ExternalConfig(_filename, _section)

__all__ = []

for key in _config:
    __all__.append(key)
    value = _config[key]
    setattr(_this_module, key, value)

}}}

Now importing * from {{{production_settings.py}}} will import data from the config file

== readthedocs.org's method ==

The `settings` directory used at [http://readthedocs.org/ Read The Docs] is available [https://github.com/rtfd/readthedocs.org/tree/master/readthedocs/settings in their repository].

== modular method ==

Move settings into a package, split everything into modules, then import in `__init__.py`.

See this [http://usrshare.net/post/28744449338/a-fairly-low-ceremony-method-for-modularizing-django post] and the [https://github.com/samiconductor/modularsettings/tree/master/modularsettings/settings example on github].

== django-flexisettings ==

Another option we use at ledapei.com is [https://github.com/hrbonz/django-flexisettings django-flexisettings] which breaks down configuration files in a very easy to use way. The stress is put on flexibility and powerful configuration possibilities, the project is then easy to share between multiple environments/developers. For instructions read the [https://github.com/hrbonz/django-flexisettings#installation installation document].

== django-split-settings ==

See the [http://www.2general.com/blog/2013/05/17/django_settings_split.html blog post] and [https://github.com/2general/django-split-settings GitHub project page] for details. Main features:

 * Settings can be split into files and directories.
 * Files later in the list can modify configurations, for example add or remove apps from INSTALLED_APPS.
 * The main settings file lists the files that make up the project’s settings.
 * Files can be marked optional. Optional files can be used to override settings per instance.
 * Wildcards can be used in file paths.
 * Maintains support for Django’s runserver auto-reloading.