Opened 5 years ago

Closed 5 years ago

Last modified 4 years ago

#12734 closed (fixed)

defer() then save() of model with custom field results in data corruption

Reported by: sachmonkey Owned by: nobody
Component: Database layer (models, ORM) Version: 1.1
Severity: Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

I have a custom field in a model (in this case JSONField, but I also reproduced this with PickledObjectField). If I run a query on that model in which I defer() the custom field and then save() that model, the custom field value gets corrupted. The models.py and tests.py code below reproduces this issue.

When the save() is called, Django retrieves the deferred field through a select statement. But when it executes the update statement, it incorrectly stores that field back into the database. If you look at the value of the data column, it goes from [1,2,3] to "[1,2,3]" (yes, the change is the quotes).

#models.py
from django.db import models
from django.utils import simplejson

class JSONField(models.TextField):
    __metaclass__ = models.SubfieldBase
    
    description = "JSONField automatically serializes\deserializes values to\from JSON."
    
    def to_python(self, value):
        if value == "":
            return None
        
        if isinstance(value, basestring):
            value = simplejson.loads(value)
        return value
 
    def get_db_prep_save(self, value):
        if value is None: return
        return simplejson.dumps(value)
 
    def value_to_string(self, obj):
        value = self._get_val_from_obj(obj)
        return self.get_db_prep_value(value)

class CustomFieldModel(models.Model):
    name = models.CharField(max_length=100)
    data = JSONField()

#tests.py
from django.test import TestCase
from models import CustomFieldModel

class DeferRegressionTest(TestCase):
    def test_defer_then_save_custom_field(self):
        #store a simple list in the basic JSON field
        test_model = CustomFieldModel()
        test_model.name = "test"
        test_model.data = [1, 2, 3]
        test_model.save()
    
        #at this point, data is of list type
        self.assertEqual(type(test_model.data), type(list()))
        
        #defer the custom JSON field
        retrieved_model = CustomFieldModel.objects.defer("data").get(name="test")
        
        #on save, django runs a select to get the deferred field and then updates it, but messes up the update statement for the deferred field
        retrieved_model.save()
        
        #now let's retrieve the whole model again
        retrieved_again_model = CustomFieldModel.objects.get(name="test")
        
        #when we get the value of the JSON field, it is no longer a list! it's been corrupted!
        self.assertEqual(type(retrieved_again_model.data), type(list()))

Attachments (1)

django-defer-save.diff (9.9 KB) - added by Alex 5 years ago.

Download all attachments as: .zip

Change History (6)

comment:1 Changed 5 years ago by Alex

  • milestone set to 1.2
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

comment:2 Changed 5 years ago by russellm

  • Component changed from Uncategorized to Database layer (models, ORM)

Changed 5 years ago by Alex

comment:3 Changed 5 years ago by jkocherhans

  • Resolution set to fixed
  • Status changed from new to closed

(In [12579]) Fixed #12734. Deferred fields will now be properly converted to python when accessed. Thanks, Alex Gaynor.

comment:4 Changed 5 years ago by jkocherhans

(In [12692]) [1.1.X] Fixed #12734. Deferred fields will now be properly converted to python when accessed. Backport of r12579 from trunk.

comment:5 Changed 4 years ago by jacob

  • milestone 1.2 deleted

Milestone 1.2 deleted

Note: See TracTickets for help on using tickets.
Back to Top