Opened 8 years ago

Closed 8 years ago

Last modified 7 years ago

#12734 closed (fixed)

defer() then save() of model with custom field results in data corruption

Reported by: sachmonkey Owned by: nobody
Component: Database layer (models, ORM) Version: 1.1
Severity: Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:


I have a custom field in a model (in this case JSONField, but I also reproduced this with PickledObjectField). If I run a query on that model in which I defer() the custom field and then save() that model, the custom field value gets corrupted. The and code below reproduces this issue.

When the save() is called, Django retrieves the deferred field through a select statement. But when it executes the update statement, it incorrectly stores that field back into the database. If you look at the value of the data column, it goes from [1,2,3] to "[1,2,3]" (yes, the change is the quotes).
from django.db import models
from django.utils import simplejson

class JSONField(models.TextField):
    __metaclass__ = models.SubfieldBase
    description = "JSONField automatically serializes\deserializes values to\from JSON."
    def to_python(self, value):
        if value == "":
            return None
        if isinstance(value, basestring):
            value = simplejson.loads(value)
        return value
    def get_db_prep_save(self, value):
        if value is None: return
        return simplejson.dumps(value)
    def value_to_string(self, obj):
        value = self._get_val_from_obj(obj)
        return self.get_db_prep_value(value)

class CustomFieldModel(models.Model):
    name = models.CharField(max_length=100)
    data = JSONField()
from django.test import TestCase
from models import CustomFieldModel

class DeferRegressionTest(TestCase):
    def test_defer_then_save_custom_field(self):
        #store a simple list in the basic JSON field
        test_model = CustomFieldModel() = "test" = [1, 2, 3]
        #at this point, data is of list type
        self.assertEqual(type(, type(list()))
        #defer the custom JSON field
        retrieved_model = CustomFieldModel.objects.defer("data").get(name="test")
        #on save, django runs a select to get the deferred field and then updates it, but messes up the update statement for the deferred field
        #now let's retrieve the whole model again
        retrieved_again_model = CustomFieldModel.objects.get(name="test")
        #when we get the value of the JSON field, it is no longer a list! it's been corrupted!
        self.assertEqual(type(, type(list()))

Attachments (1)

django-defer-save.diff (9.9 KB) - added by Alex Gaynor 8 years ago.

Download all attachments as: .zip

Change History (6)

comment:1 Changed 8 years ago by Alex Gaynor

milestone: 1.2
Triage Stage: UnreviewedAccepted

comment:2 Changed 8 years ago by Russell Keith-Magee

Component: UncategorizedDatabase layer (models, ORM)

Changed 8 years ago by Alex Gaynor

Attachment: django-defer-save.diff added

comment:3 Changed 8 years ago by jkocherhans

Resolution: fixed
Status: newclosed

(In [12579]) Fixed #12734. Deferred fields will now be properly converted to python when accessed. Thanks, Alex Gaynor.

comment:4 Changed 8 years ago by jkocherhans

(In [12692]) [1.1.X] Fixed #12734. Deferred fields will now be properly converted to python when accessed. Backport of r12579 from trunk.

comment:5 Changed 7 years ago by Jacob

milestone: 1.2

Milestone 1.2 deleted

Note: See TracTickets for help on using tickets.
Back to Top