Changes between Initial Version and Version 1 of DynamicModels

Timestamp:

May 9, 2007, 2:45:14 PM (18 years ago)

Author:

Marty Alchin <gulopine@…>

Comment:

--

DynamicModels

@@ +1 @@
+
+= Dynamic models =
+
+[[TOC]]
+
+As of [5163], Django models can now be created dynamically at run-time, rather than being defined in a Python source file. While this may seem useless on the surface, it actually provides a powerful way to prototype a Django project, even by users who don't know a thing about Python! This page lists the basic technique involved in creating models at run-time, as well as some examples of how it could be used. It is not meant to be exhaustive, nor is it meant to be drop-in code for your own project. Read and learn, don't copy.
+
+== The basic technique ==
+
+Internally, Django uses metaclasses to create models based on a class you provide in your source code. Without getting into too many details, that means that rather than your classes being the actual models, Django receives a description of your class, which it uses to create a model in its place.
+
+Thankfully, Python treats classes like any other object, so we can force this process to occur even when the class wasn't defined in source. All we really need is to give Django a description of a class, and let it do all the work of creating the model for us. Here's some bare-bones code.
+
+{{{
+#!python
+model = type(name, (models.Model,), attrs)
+}}}
+
+As you can see, describing a model for the purposes of metaclasses involves three pieces of information:
+
+ * The intended name of the class
+ * A tuple containing any classes it inherits from
+ * A dictionary containing the attributes of the class
+
+Python then processes that description as if it came from a normal class declaration, automatically triggering Django's metaclass along the way. In fact, these three bits of information correspond exactly to standard definitions, as shown in the following comparison.
+
+This:
+
+{{{
+#!python
+class Person(models.Model):
+    first_name = models.CharField(maxlength=255)
+    last_name = models.CharField(maxlength=255)
+}}}
+
+is functionally equivalent to this:
+
+{{{
+#!python
+Person = type('Person', (models.Model,), {
+    'first_name': models.CharField(maxlength=255)
+    'last_name': models.CharField(maxlength=255)
+})
+}}}
+
+Yes, that is slightly more code than the standard technique, but the advantages of not having to declare a model in Python source should become clear shortly.
+
+== A general-purpose approach ==
+
+To illustrate in a simple manner how easy it can be to create full models on the fly, consider the following function.
+
+{{{
+#!python
+def create_model(name, fields=None, app_label='', module='', options=None, admin=None):
+    "One example of how to create a model dynamically at run-time"
+
+    class Meta:
+        # Using type('Meta', ...) gives a dictproxy error during model creation
+        pass
+
+    if app_label:
+        # app_label must be set using the Meta inner class
+        setattr(Meta, 'app_label', app_label)
+
+    # Update Meta with any options that were provided
+    if options is not None:
+        for key, value in options.items():
+            setattr(Meta, key, value)
+
+    # Set up a dictionary to simulate declarations within a class
+    attrs = {'__module__': module, 'Meta': Meta}
+
+    # Add in any fields that were provided
+    if fields:
+        attrs.update(fields)
+
+    # Create an Admin inner class if admin options were provided
+    if admin is not None:
+        class Admin:
+            pass
+        for key, value in admin:
+            setattr(Admin, key, value)
+        attrs['Admin'] = Admin
+
+    # Create the class, which automatically triggers ModelBase processing
+    return type(name, (models.Model,), attrs)
+}}}
+
+This function provides everything necessary to make a fully functional Django model from scratch, even given data that's not available until the application is up and running. The arguments it takes work as follows:
+
+ * {{{name}}} - The name of the model to be created
+ * {{{fields}}} - A dictionary of fields the model will have (managers and methods would go in this dictionary as well)
+ * {{{app_label}}} - A custom application label for the model (this does not have to exist in your project, but see the [[#Admininterface Admin drawback]] below)
+ * {{{module}}} - An arbitrary module name to use as the model's source (prior to [5163], this had to be a real module path that had in fact been loaded)
+ * {{{options}}} - A dictionary of options, as if they were provided to the inner {{{Meta}}} class
+ * {{{admin}}} - A dictionary of admin options, as if they were provided to the inner {{{Admin}}} class (again, see [[#Admininterface Admin drawback]] below)
+
+Models can be created using this class with any number of features, as shown by the examples below.
+
+{{{
+#!python
+>>> model = create_model('Empty')
+>>> model.__module__
+''
+>>> model._meta.app_label
+''
+>>> len(model._meta.fields) # Remember, an "id" field is created automatically for each model
+1
+}}}
+
+{{{
+#!python
+>>> fields = {
+...     'first_name': models.CharField(maxlength=255),
+...     'last_name': models.CharField(maxlength=255),
+...     '__str__': lambda self: '%s %s' (self.first_name, self.last_name),
+... }
+>>> options = {
+...     'ordering' = ['last_name', 'first_name'],
+...     'verbose_name' = 'valued customer',
+... }
+>>> admin = {} # An empty dictionary is equivalent to "class Admin: pass"
+>>> model = create_model('Person', fields,
+...     options=options,
+...     admin=admin,
+...     app_label='fake_app',
+...     module='fake_project.fake_app.no_models',
+... )
+>>> model._meta.verbose_name_plural
+'valued customers'
+>>> len(model._meta.fields) # Remember, an "id" field is created automatically for each model
+3
+}}}
+
+== A database-driven approach ==
+
+Much more useful, however, is the ability to manage models using data contained in other models. This would allow model prototypes to be created and shared using the standard admin interface, even by users with no Python knowledge. Consider the following models.
+
+{{{
+#!python
+from django.core.validators import ValidationError
+
+class App(models.Model):
+    name = models.CharField(maxlength=255)
+    module = models.CharField(maxlength=255)
+
+    def __str__(self):
+        return self.name
+
+class Model(models.Model):
+    app = models.ForeignKey(App, related_name='models')
+    name = models.CharField(maxlength=255)
+
+    def __str__(self):
+        return self.name
+
+    def get_django_model(self):
+        "Returns a functional Django model based on current data"
+        # Get all associated fields into a list ready for dict()
+        fields = [(f.name, f.get_django_field()) for f in self.fields.all()]
+
+        # Use the create_model function defined above
+        return create_model(self.name, dict(fields), self.app.name, self.app.module)
+
+    class Meta:
+        unique_together = (('app', 'name'),)
+
+def is_valid_field(self, field_data, all_data):
+    if hasattr(models, field_data) and issubclass(getattr(models, field_data), models.Field):
+        # It exists and is a proper field type
+        return
+    raise ValidationError("This is not a valid field type.")
+
+class Field(models.Model):
+    model = models.ForeignKey(Model, related_name='fields')
+    name = models.CharField(maxlength=255)
+    type = models.CharField(maxlength=255, validator_list=[is_valid_field])
+
+    get_django_field(self):
+        "Returns the correct field type, instantiated with applicable settings"
+        # Get all associated settings into a list ready for dict()
+        settings = [(s.name, s.value) for s in field.settings.all()]
+
+        # Instantiate the field with the settings as **kwargs
+        attrs[field.name] = getattr(models, field.name)(**dict(settings))
+
+    class Meta:
+        unique_together = (('model', 'name'),)
+
+class Setting(models.Model):
+    field = models.ForeignKey(field, related_name='settings')
+    name = models.CharField(maxlength=255)
+    value = models.CharField(maxlength=255)
+
+    class Meta:
+        unique_together = (('field', 'name'),)
+}}}
+
+This doesn't take {{{Meta}}} or {{{Admin}}} options into account, nor does it allow for the creation of any model methods, but it's a decent example of how it could be done. Now a model can be created based on a database-backed description, as simply as this (but see the [[#Syncdb syncdb]] section for more details):
+
+{{{
+#!python
+>>> model = Model.objects.get(app__name='fake_project', name='FakeModel')
+>>> model
+<Model: FakeModel>
+>>> model._meta.app_label
+'fake_project'
+>>> model.objects.count()
+45
+}}}
+
+== Drawbacks ==
+
+While dynamic model creation has incredible potential, it should not be considered a recommended way to go about model declarations. Most projects won't find any value in it, and those that do have a bit of work making sure their definition scheme is robust enough for their needs. In addition to the decision of whether to use them, there are a few other details that somewhat hinder their use.
+
+=== Syncdb ===
+
+The standard {{{syncdb}}} function provided by {{{manage.py}}} relies on crawling through apps that are registered during Django's startup. This means that dynamic models (likely) won't even be found by the time {{{syncdb}}} runs. Even if a dynamic model is found and loaded, its {{{app_label}}} (whether calculated based on its {{{__module__}}} or provided through {{{Meta}}}) '''must match an entry in your {{{INSTALLED_APPS}}} setting''' in order to be processed properly.
+
+This is by design, and should not be considered a bug of any kind. It's just a fact of life when dealing with models in this manner. Without having a table in the database, however, dynamic models will be unable to perform any queries; they just sit there looking shiny. One workaround for basic models uses an internal portion of {{{django.core.management}}} to install a basic table definition to the database.
+
+{{{
+#!python
+def install(model):
+    from django.core import management
+    from django.db import connection
+
+    # Standard syncdb expects models to be in reliable locations,
+    # so dynamic models need to bypass django.core.management.syncdb.
+    # On the plus side, this allows individual models to be installed
+    # without installing the entire project structure.
+    # On the other hand, this means that things like relationships and
+    # indexes will have to be handled manually.
+    # This installs only the basic table definition.
+
+    cursor = connection.cursor()
+    statements, pending = management._get_sql_model_create(model)
+    for sql in statements:
+        cursor.execute(sql)
+}}}
+
+As the comments indicate, this will only create the single table definition, so proceed with caution if you use this function. For a production environment, a more robust solution should be found.
+
+<speculation type="wild">Since this involves {{{app_label}}}, #3591 ''might'' help, but more research is necessary on that.</speculation>
+
+=== Admin interface ===
+
+Django's built-in admin interface also relies on your project's {{{INSTALLED_APPS}}} setting, so models created dynamically will not show up in the admin unless their {{{app_label}}}s match up with packages listed in {{{INSTALLED_APPS}}}. This is again by design, and should not be considered a bug. Applications which make use of dynamic models should either create a placeholder app and put it in the project settings, or create a custom data interface for accessing the dynamic models.
+
+{{{
+#!sh
+manage.py startapp my_placeholder
+}}}
+
+{{{
+#!python
+INSTALLED_APPS = (
+    ...
+    ('myproject.myapp.my_placeholder'),
+    ...
+)
+}}}
+
+{{{
+#!python
+model = create_model('Model', app_label='my_placeholder')
+}}}
+
+'''HINT:''' Use a placeholder app unless you ''really'' need more flexibility.