[[PageOutline]] = Splitting up the settings file = If you use a source control system (CVS, SVN, ...), or want to publish your application on the web, it may be a good idea to move sensitive or machine/user specific settings like database passwords and such out of the main settings.py file. As discussions on the [http://groups.google.com/group/django-developers django-developers mailing list] have shown everybody has different requirements and ideas how to do this. This page is meant to collect some of these ideas for future reference. One thing to keep in mind is that Django's config files are pure Python. This gives you the ultimate flexibility to handle configurations the way you think is best. Or to quote Adrian Holovaty: We don't need a default solution for this. It's not within the scope of this project to tell people how they should organize their settings files. Take that opportunity to showcase your individualism. == Links to discussions == * [http://stackoverflow.com/questions/2035733 how to modularize django settings.py?] on stackoverflow.com == Different settings in different files == I believe the first user who came up with this was Hugo, who used this method in the projects he published on [https://simon.bofh.ms/cgi-bin/trac-django-projects.cgi/ his site]. {{{ #!python SECRET_KEY = open(os.path.expanduser('~/.gallery-secret')).read().strip() }}} == ini-style file for deployment == This is a solution that Michael Radziej posted to django-developers. His motivation was to be able to store the settings for an app he published under `/etc` with all the other system config files. For a similar but more transparent solution see [[SplitSettings#foxsmethod|fox's method]] `/etc/whatever/settings.ini` {{{ [database] DATABASE_USER: bla DATABASE_PASSWORD: XXXXXXXX DATABASE_HOST: dev DATABASE_PORT: DATABASE_ENGINE: mysql DATABASE_NAME: blo TESTSUITE_DATABASE_NAME: test_blo [secrets] SECRET_KEY: random-string-of-ascii CSRF_MIDDLEWARE_SECRET: random-string-of-ascii [cookies] SESSION_COOKIE_DOMAIN: # all settings in debug section should be false in productive environment # INTERNAL_IPS should be empty in productive environment [debug] DEBUG: true TEMPLATE_DEBUG: true VIEW_TEST: true INTERNAL_IPS: 127.0.0.1 SKIP_CSRF_MIDDLEWARE: true [email] SERVER_EMAIL: django@localhost EMAIL_HOST: localhost # the [error mail] and [404 mail] sections are special. Just add lines with # full name: email_address@domain.xx # each section must be present but may be empty. [error mail] Adam Smith: adam@localhost [404 mail] John Wayne: john@localhost }}} `/path/to/whatever/settings.py` {{{ #!python from ConfigParser import RawConfigParser config = RawConfigParser() config.read('/etc/whatever/settings.ini') DATABASE_USER = config.get('database', 'DATABASE_USER') DATABASE_PASSWORD = config.get('database', 'DATABASE_PASSWORD') DATABASE_HOST = config.get('database', 'DATABASE_HOST') DATABASE_PORT = config.get('database', 'DATABASE_PORT') DATABASE_ENGINE = config.get('database', 'DATABASE_ENGINE') DATABASE_NAME = config.get('database', 'DATABASE_NAME') TEST_DATABASE_NAME = config.get('database', 'TESTSUITE_DATABASE_NAME') SECRET_KEY = config.get('secrets','SECRET_KEY') CSRF_MIDDLEWARE_SECRET = config.get('secrets', 'CSRF_MIDDLEWARE_SECRET') SESSION_COOKIE_DOMAIN = config.get('cookies','SESSION_COOKIE_DOMAIN') DEBUG = config.getboolean('debug','DEBUG') TEMPLATE_DEBUG = config.getboolean('debug','TEMPLATE_DEBUG') VIEW_TEST = config.getboolean('debug', 'VIEW_TEST') INTERNAL_IPS = tuple(config.get('debug', 'INTERNAL_IPS').split()) if config.getboolean('debug', 'SKIP_CSRF_MIDDLEWARE'): MIDDLEWARE_CLASSES = tuple([x for x in list(MIDDLEWARE_CLASSES) if not x.endswith('CsrfMiddleware')]) SERVER_EMAIL = config.get('email', 'SERVER_EMAIL') EMAIL_HOST = config.get('email', 'EMAIL_HOST') ADMINS = tuple(config.items('error mail')) MANAGERS = tuple(config.items('404 mail')) }}} == Multiple setting files importing from each other == This is my (Steven Armstrong) preferred solution. Keep application wide, unsensitive settings and sane defaults in your normal `settings.py` file. /path/to/whatever/settings.py {{{ #!python import os BASE_DIR = os.path.dirname(os.path.abspath(__file__)) # why is this here? #PROJECT_DIR = os.path.abspath(os.path.join(BASE_DIR, '..')) PROJECT_DIR = BASE_DIR DEBUG = False TEMPLATE_DEBUG = DEBUG ADMINS = ( ('Mr Sysadmin', 'sysadmin@domain.tld'), ) MANAGERS = ADMINS DATABASE_ENGINE = 'sqlite3' DATABASE_NAME = os.path.join(PROJECT_DIR, 'project.db') DATABASE_USER = '' DATABASE_PASSWORD = '' DATABASE_HOST = '' DATABASE_PORT = '' TIME_ZONE = 'Europe/Zurich' LANGUAGE_CODE = 'en-us' SECRET_KEY = 'secret' #[more default and app wide settings] from settings_local import * }}} At the end of your normal `settings.py` include * from an other, machine specific config file which could look something like this. `/path/to/whatever/settings_local.py` {{{ #!python DEBUG = True TEMPLATE_DEBUG = DEBUG # don't want emails while developing ADMINS = () MANAGERS = ADMINS DATABASE_ENGINE = 'mysql' DATABASE_NAME = 'mydbname' DATABASE_USER = 'mydbuser' DATABASE_PASSWORD = 'mydbpassword' DATABASE_HOST = 'localhost' DATABASE_PORT = '' SECRET_KEY = 'random-string-of-ascii' #[more user/machine specific settings] }}} `urls.py` {{{ import settings (r'^static/(?P.*)$', 'django.views.static.serve', {'document_root': settings.BASE_DIR+'/core/static/', 'show_indexes': True}), }}} `settings.py` goes into CVS, SVN, (put your favorite RCS here)[[BR]] `settings_local.py` does _not_ go under RCS If wanted a `settings_local.template` file can be put under version control with instructions to copy it over to settings_local.py, change it to suite the environment, and to never ever commit it to the RCS system. == Development/Machine Dependant Settings Configuration == I personally use this solution. `/path/to/project/settings.py` {{{ #!python # Django project settings loader import os ROOT_PATH = os.path.dirname(os.path.abspath(__file__)) # You can key the configurations off of anything - I use project path. configs = { '/path/to/user1/dev/project': 'user1', '/path/to/user2/dev/project': 'user2', '/path/to/qa/project': 'qa', '/path/to/prod/project': 'prod', } # Import the configuration settings file - REPLACE projectname with your project config_module = __import__('config.%s' % configs[ROOT_PATH], globals(), locals(), 'projectname') # Load the config settings properties into the local scope. for setting in dir(config_module): if setting == setting.upper(): locals()[setting] = getattr(config_module, setting) }}} This settings loader will import the appropriate settings module. I store my settings files in a `config` directory in the root of the project. `/path/to/project/config/prod.py` {{{ #!python DEBUG = False TEMPLATE_DEBUG = DEBUG # don't want emails while developing ADMINS = () MANAGERS = ADMINS DATABASE_ENGINE = 'mysql' DATABASE_NAME = 'mydbname' DATABASE_USER = 'mydbuser' DATABASE_PASSWORD = 'mydbpassword' DATABASE_HOST = 'localhost' DATABASE_PORT = '' SECRET_KEY = 'random-string-of-ascii' # More production settings }}} `/path/to/project/config/user1.py` {{{ #!python # Load the production settings that we will overwrite as needed in our user1 settings file. from projectname.config.prod import * DEBUG = True DATABASE_NAME = 'devdbname' # More development/maching specific settings }}} == Using a list of conf files (Transifex) == [http://transifex.org/ Transifex] has adopted a solution using an ordered list of multiple `.conf` files inside a settings directory. This helps separating the config options, packaging in Linux distributions, and putting the settings files under `/etc`. {{{ #!python $ ls settings/ 10-base.conf 20-engines.conf 30-site.conf ... $ cat settings.py import os.path import glob conffiles = glob.glob(os.path.join(os.path.dirname(__file__), 'settings', '*.conf')) conffiles.sort() for f in conffiles: execfile(os.path.abspath(f)) }}} To apply local-only changes, create a `XX-local.conf file`, which can be ignored with your VCS of choice: {{{ #!python $ cat settings/51-local.conf INSTALLED_APPS += ['debug_toolbar'] $ cat .hgignore syntax: glob settings/*-local.conf }}} == Accessing and modifying existing settings == I ([http://djangopeople.net/akaihola Antti Kaihola]) use the following solution for defining local settings. It provides access to the original settings while defining local settings. Note: this only works if you import from the base {{{settings.py}}} or {{{settings/__init__.py}}}. {{{settings/__init__.py}}}: {{{ INSTALLED_APPS = ('whatever',) try: from settings.local import * except ImportError: pass }}} {{{settings/local.py}}}: {{{ import sys globals().update(vars(sys.modules['settings'])) INSTALLED_APPS += ('another_app',) }}} Older, clumsier methods I've used: {{{settings.py}}}: {{{ INSTALLED_APPS = ('whatever',) import more_settings more_settings.modify(globals()) }}} {{{more_settings.py}}}: {{{ def modify(settings): settings['INSTALLED_APPS'] += ('another_app',) }}} alternate more_settings.py with more magic: {{{ def config(INSTALLED_APPS=(), **other_settings): INSTALLED_APPS += ('another_app',) del other_settings return locals() def modify(settings): settings.update(config(**settings)) }}} == Settings inheritance == I ([http://djangopeople.net/cordis Sergey Yushckeyev]) have extended proposed solution: {{{env}}} {{{ export APP_SETTINGS = 'development.cordis' }}} {{{settings/__init__.py}}} {{{ #!python CONFIGURATION = 'common' def deep_update(from_dict, to_dict): for (key, value) in from_dict.iteritems(): if key in to_dict.keys() and isinstance(to_dict[key], dict) and isinstance(value, dict): deep_update(value, to_dict[key]) else: to_dict[key] = value try: modules = os.environ.get('APP_SETTINGS', '').split('.') current = __name__ for name in modules: module = getattr(__import__(current, globals(), locals(), [name]), name) current += '.' + name current_settings = {} for setting in dir(module): if setting == setting.upper(): current_settings[setting] = getattr(module, setting) deep_update(current_settings, locals()) except ImportError, e: print 'Unable to import configuration: %s' % e }}} {{{settings/development/cordis.py}}} {{{ #!python CONFIGURATION = 'cordis' }}} {{{app/models.py}}} {{{ #!python from settings import CONFIGURATION print CONFIGURATION # prints 'cordis' }}} === Setting Inheritance with Hierarchy === I ([http://twitter.com/#!/karim_a_nassar Karim A. Nassar]) need to be able to inherit from a base settings file, then for each environment provide overrides. In dev env, we need developer specific settings. The following extends [http://djangopeople.net/cordis Sergey Yushckeyev's] solution above. This script merges {{{settings/common.py}}}, {{{settings/$APP_ENV.py}}} and {{{settings/.py}}}, in that order. {{{settings/__init__.py}}} {{{ #!python import os, pwd # certain keys we want to merge instead of copy merge_keys = ('INSTALLED_APPS', 'MIDDLEWARE_CLASSES') def deep_update(from_dict, to_dict): for (key, value) in from_dict.iteritems(): if key in to_dict.keys() and isinstance(to_dict[key], dict) and isinstance(value, dict): deep_update(value, to_dict[key]) elif key in merge_keys: if not key in to_dict: to_dict[key] = () to_dict[key] = to_dict[key] + from_dict[key] else: to_dict[key] = value # this should be one of prod, qa, staging, dev. Default to dev for safety. env = os.environ.get('APP_ENV', 'dev') # try to load user specific settings uid = pwd.getpwuid(os.getuid())[0] modules = ('common', env, uid) current = __name__ for module_name in modules: try: module = getattr(__import__(current, globals(), locals(), [module_name]), module_name) except ImportError, e: print 'ERROR: Unable to import %s configuration: %s' % (module_name, e) raise except AttributeError, e: if env == 'dev' and module_name == uid: print 'WARNING: Unable to import %s dev configuration: does %s.py exist?' % (module_name, module_name) else: raise # create a local copy of this module's settings module_settings = {} for setting in dir(module): # all django settings are uppercase, so this ensures we # are only processing settings from the dir() call if setting == setting.upper(): module_settings[setting] = getattr(module, setting) deep_update(module_settings, locals()) #print locals() # for debugging }}} Then, move `settings.py` to `settings/common.py`. The above defaults to {{{dev}}} env. You could pass in anything to {{{manage.py}}} via the {{{APP_ENV}}} environment variable. Common ones would be {{{prod}}}, {{{staging}}}, {{{qa}}}, etc. For each {{{APP_ENV}}}, you create a file {{{$APP_ENV.py}}}. Finally, create {{{$USER.py}}} file with developer specific overrides. Example files (excepting {{{settings/common.py}}}) below.. `settings/dev.py` {{{ #!python import os # cwd is settings. determine project path cwd = os.path.dirname(os.path.abspath(__file__)) project_path = cwd[:-9] # chop off "settings/" DEBUG = True TEMPLATE_DEBUG = DEBUG TEMPLATE_DIRS = ( # Put strings here, like "/home/html/django_templates" or "C:/www/django/templates". # Always use forward slashes, even on Windows. # Don't forget to use absolute paths, not relative paths. # automatically inject project templates directory into the dev env '%s/templates' % ( project_path ) ) }}} `settings/$USER.py` {{{ #!python DATABASES = { 'default': { 'USER': 'me', 'PASSWORD': 'i<3django', } } INSTALLED_APPS = ( 'haystack', ) HAYSTACK_CONNECTIONS = { 'default': { 'ENGINE': 'haystack.backends.solr_backend.SolrEngine', 'URL': 'http://127.0.0.1:8983/solr' }, } }}} Of course, you may find that you actually want some of the above settings in `common.py`. Configure however you need for your situation. === Simple Package Organization for Environments === [http://chesmart.in I] have used a Python package organization similar to those above for per-environment settings, but much simplified: leaving inheritance to be handled by plain Python modules, without deep-merging dictionary magic. Just be explicit! It's easier to know what {{{production.py}}} configures at a glance without having to go dig into data structures in other files. Having per-developer settings is asking for issues with inconsistent configurations to crop up in deployment environments. {{{ #!sh [myapp]$ tree settings settings ├── __init__.py ├── defaults.py ├── dev.py └── production.py }}} The initial {{{settings.py}}} was moved to {{{settings/defaults.py}}}. By default, the settings package will use development configuration, so that local dev Just Works as normal: {{{settings/__init__.py}}} {{{ #!python from dev import * }}} {{{settings/dev.py}}} {{{ #!python # Load defaults in order to then add/override with dev-only settings from defaults import * DEBUG = True # ... etc. }}} For production (or staging, etc.), just add an appropriate module and set Django's conventional [https://docs.djangoproject.com/en/dev/topics/settings/ DJANGO_SETTINGS_MODULE environment variable] in your deployment setup to load it: {{{settings/production.py}}} {{{ #!python # Load defaults in order to then add/override with production-only settings from defaults import * DEBUG = False # ... etc. }}} Wherever appropriate for your deployment (such as an Apache {{{SetEnv}}} directive, include the equivalent of: {{{ #!sh $ export DJANGO_SETTINGS_MODULE=myapp.settings.production }}} Using {{{django-admin.py}}} in deployment rather than {{{manage.py}}} should automatically put your app's package on {{{sys.path}}} so that an import path qualified this way will work. If you must use a {{{settings/local.py}}} for some reason, virtualenvwrapper's {{{postactivate/postdeactivate}}} hooks are a good place to set {{{DJANGO_SETTINGS_MODULE}}} to point to it for local development per-project. It's advisable to eliminate as many potential discrepancies between dev and production configuration as possible, though. == Yipit's method == [http://tech.yipit.com/author/adam-nelson/ Adam Nelson] from [http://yipit.com/ Yipit] blogged about their method of managing Django settings. See [http://tech.yipit.com/2011/11/02/django-settings-what-to-do-about-settings-py/ Extending Django Settings for the Real World]. TODO: summarize the blog post here. == Rob Golding's method == Rob Golding posted on his blog how he solved this issue. It's a bit of a hack really but it works nicely and allows you to override or extend anything in {{{settings.py}}}. Add this to the bottom of {{{settings.py}}}: {{{ #!python try: LOCAL_SETTINGS except NameError: try: from local_settings import * except ImportError: pass }}} Now create a {{{local_settings.py}}} with the following: {{{ #!python LOCAL_SETTINGS = True from settings import * }}} You can now add things to {{{local_settings.py}}} like this: {{{ #!python MIDDLEWARE_CLASSES += ('debug_toolbar.middleware.DebugToolbarMiddleware',) INTERNAL_IPS = ('127.0.0.1',) INSTALLED_APPS += ('debug_toolbar',) }}} This does however have the drawback of {{{settings.py}}} being parsed twice but that shouldn't cause any sort of mention worthy performance loss. == fox's method == First write a config file with passwords ad secret keys. Save it somewhere. This is the `external_config.py` {{{ #!python from ConfigParser import ConfigParser, NoOptionError class ExternalConfig(object): """ Wrapper for ConfigParser Reads a section from a config file and provides an object with the properties and the values defined in the config. """ def __init__(self, conf_file, section): """ Reads the section "section" from the config "conf_file" """ self._conf_file = conf_file self._section = section self._config = ConfigParser() self._config.read(self._conf_file) self._index = len(self._config.options(self._section)) def __getattr__(self, name): """ Read the property from the config """ if name.startswith('_'): return self.__dict__[name] try: value = self._config.get(self._section, name) if value.isdigit(): return int(value) elif value in ('True', 'False'): return value == 'True' else: return value except NoOptionError: raise AttributeError def __getitem__(self, name): """ Read the property from the config """ return self.__getattr__(name) def __iter__(self): """ __iter__ method """ return self def next(self): """ next method """ if self._index == 0: raise StopIteration self._index -= 1 return self._config.options(self._section)[self._index] }}} This is `production_settings.py` {{{ #!python from external_config import ExternalConfig import sys _filename = "" # add your config file _section = "mysection" # add your config section _this_module = sys.modules[__name__] _config = ExternalConfig(_filename, _section) __all__ = [] for key in _config: __all__.append(key) value = _config[key] setattr(_this_module, key, value) }}} Now importing * from {{{production_settings.py}}} will import data from the config file