Dynamic models

TOC

As of [5163], Django models can now be created dynamically at run-time, rather than being defined in a Python source file. While this may seem useless on the surface, it actually provides a powerful way to prototype a Django project, even by users who don't know a thing about Python! This page lists the basic technique involved in creating models at run-time, as well as some examples of how it could be used. It is not meant to be exhaustive, nor is it meant to be drop-in code for your own project. Read and learn, don't copy.

The basic technique

Internally, Django uses metaclasses to create models based on a class you provide in your source code. Without getting into too many details, that means that rather than your classes being the actual models, Django receives a description of your class, which it uses to create a model in its place. For more details, see the writeup on Model Creation.

Thankfully, Python treats classes like any other object, so we can force this process to occur even when the class wasn't defined in source. All we really need is to give Django a description of a class, and let it do all the work of creating the model for us. Here's some bare-bones code.

model = type(name, (models.Model,), attrs)

As you can see, describing a model for the purposes of metaclasses involves three pieces of information:

• The intended name of the class
• A tuple containing any classes it inherits from
• A dictionary containing the attributes of the class

Python then processes that description as if it came from a normal class declaration, automatically triggering Django's metaclass along the way. In fact, these three bits of information correspond exactly to standard definitions, as shown in the following comparison.

This:

class Person(models.Model):
    first_name = models.CharField(maxlength=255)
    last_name = models.CharField(maxlength=255)

is functionally equivalent to this:

Person = type('Person', (models.Model,), {
    'first_name': models.CharField(maxlength=255),
    'last_name': models.CharField(maxlength=255),
})

Yes, that is slightly more code than the standard technique, but the advantages of not having to declare a model in Python source should become clear shortly.

A general-purpose approach

To illustrate in a simple manner how easy it can be to create full models on the fly, consider the following function.

def create_model(name, fields=None, app_label='', module='', options=None, admin=None):
    "One example of how to create a model dynamically at run-time"

    class Meta:
        # Using type('Meta', ...) gives a dictproxy error during model creation
        pass

    if app_label:
        # app_label must be set using the Meta inner class
        setattr(Meta, 'app_label', app_label)

    # Update Meta with any options that were provided
    if options is not None:
        for key, value in options.items():
            setattr(Meta, key, value)

    # Set up a dictionary to simulate declarations within a class
    attrs = {'__module__': module, 'Meta': Meta}

    # Add in any fields that were provided
    if fields:
        attrs.update(fields)

    # Create an Admin inner class if admin options were provided
    if admin is not None:
        class Admin:
            pass
        for key, value in admin:
            setattr(Admin, key, value)
        attrs['Admin'] = Admin

    # Create the class, which automatically triggers ModelBase processing
    return type(name, (models.Model,), attrs)

This function provides everything necessary to make a fully functional Django model from scratch, even given data that's not available until the application is up and running. The arguments it takes work as follows:

• name - The name of the model to be created
• fields - A dictionary of fields the model will have (managers and methods would go in this dictionary as well)
• app_label - A custom application label for the model (this does not have to exist in your project, but see the Admin drawback below)
• module - An arbitrary module name to use as the model's source (prior to [5163], this had to be a real module path that had in fact been loaded)
• options - A dictionary of options, as if they were provided to the inner Meta class
• admin - A dictionary of admin options, as if they were provided to the inner Admin class (again, see Admin drawback below)

Models can be created using this class with any number of features, as shown by the examples below.

>>> model = create_model('Empty')
>>> model.__module__
''
>>> model._meta.app_label
''
>>> len(model._meta.fields) # Remember, an "id" field is created automatically for each model
1

>>> fields = {
...     'first_name': models.CharField(maxlength=255),
...     'last_name': models.CharField(maxlength=255),
...     '__str__': lambda self: '%s %s' (self.first_name, self.last_name),
... }
>>> options = {
...     'ordering' = ['last_name', 'first_name'],
...     'verbose_name' = 'valued customer',
... }
>>> admin = {} # An empty dictionary is equivalent to "class Admin: pass"
>>> model = create_model('Person', fields,
...     options=options,
...     admin=admin,
...     app_label='fake_app',
...     module='fake_project.fake_app.no_models',
... )
>>> model._meta.verbose_name_plural
'valued customers'
>>> len(model._meta.fields) # Remember, an "id" field is created automatically for each model
3

A database-driven approach

Much more useful, however, is the ability to manage models using data contained in other models. This would allow model prototypes to be created and shared using the standard admin interface, even by users with no Python knowledge. Consider the following models.

from django.core.validators import ValidationError

class App(models.Model):
    name = models.CharField(maxlength=255)
    module = models.CharField(maxlength=255)

    def __str__(self):
        return self.name

class Model(models.Model):
    app = models.ForeignKey(App, related_name='models')
    name = models.CharField(maxlength=255)

    def __str__(self):
        return self.name

    def get_django_model(self):
        "Returns a functional Django model based on current data"
        # Get all associated fields into a list ready for dict()
        fields = [(f.name, f.get_django_field()) for f in self.fields.all()]

        # Use the create_model function defined above
        return create_model(self.name, dict(fields), self.app.name, self.app.module)

    class Meta:
        unique_together = (('app', 'name'),)

def is_valid_field(self, field_data, all_data):
    if hasattr(models, field_data) and issubclass(getattr(models, field_data), models.Field):
        # It exists and is a proper field type
        return
    raise ValidationError("This is not a valid field type.")

class Field(models.Model):
    model = models.ForeignKey(Model, related_name='fields')
    name = models.CharField(maxlength=255)
    type = models.CharField(maxlength=255, validator_list=[is_valid_field])

    def get_django_field(self):
        "Returns the correct field type, instantiated with applicable settings"
        # Get all associated settings into a list ready for dict()
        settings = [(s.name, s.value) for s in self.settings.all()]

        # Instantiate the field with the settings as **kwargs
        return getattr(models, self.type)(**dict(settings))

    class Meta:
        unique_together = (('model', 'name'),)

class Setting(models.Model):
    field = models.ForeignKey(Field, related_name='settings')
    name = models.CharField(maxlength=255)
    value = models.CharField(maxlength=255)

    class Meta:
        unique_together = (('field', 'name'),)

This doesn't take Meta or Admin options into account, nor does it allow for the creation of any model methods, but it's a decent example of how it could be done. Now a model can be created based on a database-backed description, as simply as this (but see the syncdb section for more details):

>>> model = Model.objects.get(app__name='fake_project', name='FakeModel')
>>> model
<Model: FakeModel>
>>> model._meta.app_label
'fake_project'
>>> model.objects.count()
45

Cloning existing models

There are some situations where it's useful to copy existing models, make slight modifications, and have them available to Django. One example of this is an AuditTrail and its article describes the techniques necessary to accomplish this.

Drawbacks

While dynamic model creation has incredible potential, it should not be considered a recommended way to go about model declarations. Most projects won't find any value in it, and those that do have a bit of work making sure their definition scheme is robust enough for their needs. In addition to the decision of whether to use them, there are a few other details that somewhat hinder their use.

Syncdb

The standard syncdb function provided by manage.py relies on crawling through apps that are registered during Django's startup. This means that dynamic models (likely) won't even be found by the time syncdb runs. Even if a dynamic model is found and loaded, its app_label (whether calculated based on its __module__ or provided through Meta) must match an entry in your INSTALLED_APPS setting in order to be processed properly.

This is by design, and should not be considered a bug of any kind. It's just a fact of life when dealing with models in this manner. Without having a table in the database, however, dynamic models will be unable to perform any queries; they just sit there looking shiny. One workaround for basic models uses an internal portion of django.core.management to install a basic table definition to the database.

def install(model):
    from django.core import management
    from django.db import connection

    # Standard syncdb expects models to be in reliable locations,
    # so dynamic models need to bypass django.core.management.syncdb.
    # On the plus side, this allows individual models to be installed
    # without installing the entire project structure.
    # On the other hand, this means that things like relationships and
    # indexes will have to be handled manually.
    # This installs only the basic table definition.

    # disable terminal colors in the sql statements
    management.disable_termcolors()

    cursor = connection.cursor()
    statements, pending = management._get_sql_model_create(model)
    for sql in statements:
        cursor.execute(sql)

As the comments indicate, this will only create the single table definition, so proceed with caution if you use this function. For a production environment, a more robust solution should be found.

<speculation type="wild">Since this involves app_label, #3591 might help, but more research is necessary on that.</speculation>

Admin interface

Django's built-in admin interface also relies on your project's INSTALLED_APPS setting, so models created dynamically will not show up in the admin unless their app_labels match up with packages listed in INSTALLED_APPS. This is again by design, and should not be considered a bug. Applications which make use of dynamic models should either create a placeholder app and put it in the project settings, or create a custom data interface for accessing the dynamic models.

manage.py startapp my_placeholder

INSTALLED_APPS = (
    ...
    ('myproject.myapp.my_placeholder'),
    ...
)

model = create_model('Model', app_label='my_placeholder')

HINT: Use a placeholder app unless you really need more flexibility.