Opened 14 years ago

Closed 14 years ago

Last modified 12 years ago

#12734 closed (fixed)

defer() then save() of model with custom field results in data corruption

Reported by: sachmonkey Owned by: nobody
Component: Database layer (models, ORM) Version: 1.1
Severity: Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I have a custom field in a model (in this case JSONField, but I also reproduced this with PickledObjectField). If I run a query on that model in which I defer() the custom field and then save() that model, the custom field value gets corrupted. The models.py and tests.py code below reproduces this issue.

When the save() is called, Django retrieves the deferred field through a select statement. But when it executes the update statement, it incorrectly stores that field back into the database. If you look at the value of the data column, it goes from [1,2,3] to "[1,2,3]" (yes, the change is the quotes).

#models.py
from django.db import models
from django.utils import simplejson

class JSONField(models.TextField):
    __metaclass__ = models.SubfieldBase
    
    description = "JSONField automatically serializes\deserializes values to\from JSON."
    
    def to_python(self, value):
        if value == "":
            return None
        
        if isinstance(value, basestring):
            value = simplejson.loads(value)
        return value
 
    def get_db_prep_save(self, value):
        if value is None: return
        return simplejson.dumps(value)
 
    def value_to_string(self, obj):
        value = self._get_val_from_obj(obj)
        return self.get_db_prep_value(value)

class CustomFieldModel(models.Model):
    name = models.CharField(max_length=100)
    data = JSONField()

#tests.py
from django.test import TestCase
from models import CustomFieldModel

class DeferRegressionTest(TestCase):
    def test_defer_then_save_custom_field(self):
        #store a simple list in the basic JSON field
        test_model = CustomFieldModel()
        test_model.name = "test"
        test_model.data = [1, 2, 3]
        test_model.save()
    
        #at this point, data is of list type
        self.assertEqual(type(test_model.data), type(list()))
        
        #defer the custom JSON field
        retrieved_model = CustomFieldModel.objects.defer("data").get(name="test")
        
        #on save, django runs a select to get the deferred field and then updates it, but messes up the update statement for the deferred field
        retrieved_model.save()
        
        #now let's retrieve the whole model again
        retrieved_again_model = CustomFieldModel.objects.get(name="test")
        
        #when we get the value of the JSON field, it is no longer a list! it's been corrupted!
        self.assertEqual(type(retrieved_again_model.data), type(list()))

Attachments (1)

django-defer-save.diff (9.9 KB ) - added by Alex Gaynor 14 years ago.

Download all attachments as: .zip

Change History (6)

comment:1 by Alex Gaynor, 14 years ago

milestone: 1.2
Triage Stage: UnreviewedAccepted

comment:2 by Russell Keith-Magee, 14 years ago

Component: UncategorizedDatabase layer (models, ORM)

by Alex Gaynor, 14 years ago

Attachment: django-defer-save.diff added

comment:3 by jkocherhans, 14 years ago

Resolution: fixed
Status: newclosed

(In [12579]) Fixed #12734. Deferred fields will now be properly converted to python when accessed. Thanks, Alex Gaynor.

comment:4 by jkocherhans, 14 years ago

(In [12692]) [1.1.X] Fixed #12734. Deferred fields will now be properly converted to python when accessed. Backport of r12579 from trunk.

comment:5 by Jacob, 12 years ago

milestone: 1.2

Milestone 1.2 deleted

Note: See TracTickets for help on using tickets.
Back to Top