New Manipulators

[This is section is a lot of explination, to get right to some code scroll down.] Although validation aware models are great for database integrity, and should be leveraged within the manipulator system, it is still necessary for manipulators to do most of the work. There are three main steps to the process of taking post data and updating or creating a model object with that data:

1. Gather and convert the post data.
2. Validate the converted data.
3. Apply the data to the object and save, aka do something with the data.

A validation aware model alone cannot complete this task as information needs to be provided that details the sorts of fields that should be used, and processes are needed to convert from string / post data to the pythonic data. Furthermore many manipulation and form tasks require form fields that do not map directly to a model field, for instance a password changing form that requires a password to be given twice. This example also illuminates how a form field might map directly to a model field, but may not want to give its direct value to the field. As in a password, one probably wouldn't want: user.password = datapassword, but rather: user.set_password(datapassword).

So we change Manipulators some, and then add model validation. But we need to make our manipulator system leverage it. In other words, when our model throws a ValidationError, we need to add it to our errors and then present our form back to the user. This should be made simple for the developer so that they don't have to create a try/except block every time they want to save an object.

This all means that validation exceptions could be thrown at any point in our manipulation process: conversion, validation, and model object saving. There might even be more steps in the future, or with manipulators that aren't HTML based but something else like JSON.

There are a few ways of doing this, from the developer's perspective. Currently manipulator process in a custom view for the developer is needlessly complicated. Compare the ​current process to the ​rather basic one that added to the cookbook. But with model validation it complicates both with even more code for checking for errors after the object.save().

So it occured to me that a form was a lot like exception. At various points during the manipulation process it becomes evident that user input is needed, and that we cannot continue on. This comes to a head at one point in the view where it returns a render_to_response('form_template', {'form': form}). So why not make the Form class an actual Exception? This way we can raise the form all the way back to the view. Here then is the basic form of my proposed view:

def create_article(request):
    try:
        m = Article.CreateManipulator(request)
        article = m.save()
        return HttpResponseRedirect('/articles/update/%d/' % article.id)
    except Form, form:
        return render_to_response('articles/create.html', {'form': form})

As you can see, our form is now more of an exception. And our manipulation process is very simple, most of it takes place in the constructor itself, if durring that time a form is raised, it is presented. Otherwise call "save()" on our manipulator. In this case it creates and returns a new article. It will also raise a form if the model validation returns an error. If no form is raised, we are done and return our Response.

class Manipulator(object):
    """     
    A Manipulator "manipulates" data and objects.  
    Sample usage with Article.CreateManipulator:
    
    def create_article(request):
    try:
        m = Article.CreateManipulator(request)
        article = m.save()
        return HttpResponseRedirect(article.get_update_url())
    except Form, form:
        return render_to_response('articles/create.html', {'form': form})
    
    As you can see, if a Manipulator needs to get information with a form, it 
    will raise it's form.  At that point you can display it.

    To create a custom manipulator you have four functions that you may 
    want to override, each is a step in the maniplator process:
        * __init__
        * convert
        * validate
        * save
        
    And there are four more attributes that you will deal with:
        * fields - A sequence that defines the fields of the Manipulator
        * errors - A dictionary that holds a mapping of field-name -> error list
        * data - A dictionary of the current data.  Remember that this may be
                    raw data if conversion has not yet been done.
        * form - This is actually a property that returns the form to be given
                    in the template context.
    """
    
    def __init__(self, request):
        """
        Initializes the Manipulator and instigates the process.  
        Mainly you will want to define your fields here.
        """
        self.fields = (
            SlugField("slug", attributes={'class': 'slug'}),
            TextField("title", label="Headline", maxlength=60, attributes={'class': 'title'}),
            LargeTextField("body", label="Story", attributes={'class': 'body'}),
            DateTimeField("pub_date", label="This article will publish on this date"),
        )
        self.process(request.POST)
    
    def convert(self, data):
        """
        This conversion function will happen AFTER the data is already converted 
        by the fields.  More conversion can be done here.
        """
        pass
    
    def validate(self, data):
        """
        Here you can further validation after the fields have already had their
        go at it.
        """
        pass
    
    def save(self):
        """
        Once the data has been converted and validated it is ready to be 
        processed.  Usually this will entail updating an model instance.  To aid in
        that task, there is a convenience self.update <see below>.
        """
        pass
    
    ### Utility Functions ###
    def update(self, object, data):
        """
        Updates and saves a model object.  This will raise a form if there are any errors.
        """
        self.errors = object.update(data)
        object.save()
        if (self.errors):
            raise Form
        return object
            
    def add_errors(self, field, errors):
        "A convenience function to add errors to a field's error list."
        errors = list(errors)
        self.errors.setdefault(field.name, []).extend(errors)
    
    ### Interneal functions ###
    def process(self, data):
        "Perform the manipulation process."
        if (not data):
            raise form
        self.data = data.copy()
        self.errors = {}
        self._convert()
        self._validate()
    
    def _get_form(self):
        return Form(self.fields, self.data, self.errors)
    form = property(_get_form)
    
    def _convert(self)
        """
        Goes through each field and converts the data in place on the data dictionary.
        This will raise a form if there are validation errors by the end.
        """
        for field in self.fields:
            if (field.name in self.data):
                try:
                    self.data[field.name] = field.to_python(self.data[field.field_name])
                except (validators.ValidationError, validators.CriticalValidationError), e:
                    self.errors.setdefault(field.field_name, []).extend(e.messages)
        if (self.errors):
            raise self.form
        self.convert(self.data)
    
    def _validate(self):
        """
        Goes through each field and validates the data.  This will raise a form if there are 
        validation errors by the end.
        """
        for field in self.fields:
            for validator in field.validators:
                try:
                    validator(self.data[field.name])
                except (validators.ValidationError, validators.CriticalValidationError), e:
                    self.add_errors(e.messages)
        if (self.errors):
            raise self.form
        self.validate(self.data)
    
    ### Default Properties ###
    data = {}
    errors = {}
    fields = ()

### Manipulator factories ###
def create_manipulator(model_cls):
    "Factory, making a CreateManipulator for a specified model class"
    
    class CreateManipulator(Manipulator):
        def __init__(self, request):
            self.fields = model_cls._meta.create_fields
            self.process(request.POST)
            
        def save(self):
            self.update(model_cls(), data)
            
    return CreateManipulator
   
def update_manipulator(model_cls):
    "Factory, making a UpdateManipulator for a specified model class"
    
    class UpdateManipulator(Manipulator):
        def __init__(self, request, object):
            self.object = object
            self.process(request.POST)
            self.fields = model_cls._meta.update_fields
            
        def save(self):
            self.update(self.object, data)
            
    return UpdateManipulator

### Attach to classes ###
Article.CreateManipulator = create_manipulator(Article)
Article.UpdateManipulator = update_manipulator(Article)

### Custom Manipulator ###
class CustomManipulator(Manipulator):
    def __init__(self, request):
        self.fields = (
            SlugField("slug", attributes={'class': 'slug'}),
            TextField("title", label="Headline", maxlength=60, attributes={'class': 'title'}),
            LargeTextField("body", label="Story", attributes={'class': 'body'}),
            DateTimeField("pub_date", label="This article will publish on this date"),
        )
        self.process(request.POST)
    
    def save(self):
        Article=
        pass
        
### Template Tag ###
{% render-form form %}
# or
{% render-form-row form.slug %}
# or ... etc.

### Generic Views ###
def create_view(model, template=None):
    """
    Generic create view.  Redirects to object.get_update_url() or 
    object.get_absolute_url() when it's done.
    
    TODO: Add all the other stuff already in the create view.
    """
    if template == None:
        template = "%s/%s_create.html" % (model._meta.app_label, model._meta.object_name.lower())
    def create_func(request):
        try:
            m = model.CreateManipulator(request)
            object = m.save()
            if hasattr(object, 'get_update_url'):
                return HttpResponseRedirect(object.get_update_url())
            else:
                return HttpResponseRedirect(object.get_absolute_url())
        except Form, form:
            return render_to_response(template, {'form': form})
    return create_func
    
def update_view(model, template=None):
    """
    Generic update view.  Redirects to object.get_absolute_url() when it's done.
    
    TODO: Add all the other stuff already in the update view.
    """
    if template == None:
        template = "%s/%s_update.html" % (model._meta.app_label, model._meta.object_name.lower())
    def update_func(request, pk):
        object = get_object_or_404(model, pk=pk)
        try:
            m = model.UpdateManipulator(request)
            m.save()
            return HttpResponseRedirect(object.get_absolute_url())
        except Form, form:
            return render_to_response(template, {'form': form, 'object': object})
    return update_func   

### urls.py usage ###
from app.models import Article
('^articles/create/%d/$', 'django.views.generic.create_update.create_view', {'model':Article})
('^articles/update/$',      'django.views.generic.create_update.update_view', {'model':Article})

### Custom views ###
def create_article(request):
    try:
        m = Article.CreateManipulator(request)
        article = m.save()
        return HttpResponseRedirect('/articles/update/%d/' % article.id)
    except Form, form:
        return render_to_response('articles/create.html', {'form': form})
            
def update_article(request, id):
    article = get_object_or_404(Article, pk=id)
    try:
        m = Article.UpdateManipulator(request, article)
        m.save()
        return HttpResponseRedirect('/articles/%d/' % article.id)
    except Form, form:
        return render_to_response('articles/update.html', {'form': form, 'object': object})

Compatibility

There would be some changes needed in some other django code for all of this to work out right:

• The FormWrapper class would be destroyed, I believe, and would be supplanted by Form.
• Form fields would alter somewhat, most of it would be cosmetic (e.g. changing "html2python" to "to_python" and adding a "label" and "attributes" keyword arguments in the constructor)
• Models would gain an "update" function that took a dictionary, applied it to the properties, and then validated.
• Meta in the Models would gain a "update_fields" and "create_fields" attribute. Here you could define the fields used for your model's default Manipulators.