Opened 9 years ago

Closed 9 years ago

Last modified 9 years ago

#25233 closed Bug (fixed)

Unable save model in admin with HStoreField field after manually saving invalid data

Reported by: hongphi Owned by: Tim Graham
Component: contrib.postgres Version: 1.8
Severity: Release blocker Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Tim Graham)

f = Food.objects.get(id=2)
f.weight= {'0.5': '0.5kg - 1kg', '1.5': '1.1kg - 2kg'}
f.save()

Save model by code then OK. But I can't save model in admin. See error below:

TypeError at /admin/food/food/2/
expected string or buffer
Request Method:	POST
Request URL:	http://127.0.0.1:8000/admin/food/food/2/
Django Version:	1.8.3
Exception Type:	TypeError
Exception Value:	
expected string or buffer
Exception Location:	C:\Python27\Lib\json\decoder.py in decode, line 365
Python Executable:	D:\Program Files\PythonEnv\foodgreen\Scripts\python.exe
Python Version:	2.7.6
Python Path:	
['D:\\Projects\\Django8\\greenfood',
 'D:\\Program Files\\eclipse\\plugins\\org.python.pydev_4.2.0.201507041133\\pysrc',
 'D:\\Projects\\Django8\\greenfood',
 'D:\\Program Files\\PythonEnv\\foodgreen\\lib\\site-packages\\psycopg2-2.6.1-py2.7-win32.egg',
 'D:\\Program Files\\PythonEnv\\foodgreen\\lib',
 'D:\\Program Files\\PythonEnv\\foodgreen\\Scripts',
 'C:\\Python27\\Lib',
 'C:\\Python27\\DLLs',
 'C:\\Python27\\Lib\\lib-tk',
 'D:\\Program Files\\PythonEnv\\foodgreen',
 'D:\\Program Files\\PythonEnv\\foodgreen\\lib\\site-packages',
 'C:\\WINDOWS\\SYSTEM32\\python27.zip',
 'D:\\Program Files\\PythonEnv\\foodgreen\\DLLs',
 'D:\\Program Files\\PythonEnv\\foodgreen\\lib\\plat-win',
 'D:\\Program Files\\PythonEnv\\foodgreen\\lib\\lib-tk']
Server time:	Thu, 6 Aug 2015 16:48:26 +0700
Traceback Switch to copy-and-paste view

D:\Program Files\PythonEnv\foodgreen\lib\site-packages\django\core\handlers\base.py in get_response
                                response = wrapped_callback(request, *callback_args, **callback_kwargs) ...
▶ Local vars
D:\Program Files\PythonEnv\foodgreen\lib\site-packages\django\contrib\admin\options.py in wrapper
                            return self.admin_site.admin_view(view)(*args, **kwargs) ...
▶ Local vars
D:\Program Files\PythonEnv\foodgreen\lib\site-packages\django\utils\decorators.py in _wrapped_view
                                response = view_func(request, *args, **kwargs) ...
▶ Local vars
D:\Program Files\PythonEnv\foodgreen\lib\site-packages\django\views\decorators\cache.py in _wrapped_view_func
                    response = view_func(request, *args, **kwargs) ...
▶ Local vars
D:\Program Files\PythonEnv\foodgreen\lib\site-packages\django\contrib\admin\sites.py in inner
                        return view(request, *args, **kwargs) ...
▶ Local vars
D:\Program Files\PythonEnv\foodgreen\lib\site-packages\django\contrib\admin\options.py in change_view
                    return self.changeform_view(request, object_id, form_url, extra_context) ...
▶ Local vars
D:\Program Files\PythonEnv\foodgreen\lib\site-packages\django\utils\decorators.py in _wrapper
                        return bound_func(*args, **kwargs) ...
▶ Local vars
D:\Program Files\PythonEnv\foodgreen\lib\site-packages\django\utils\decorators.py in _wrapped_view
                                response = view_func(request, *args, **kwargs) ...
▶ Local vars
D:\Program Files\PythonEnv\foodgreen\lib\site-packages\django\utils\decorators.py in bound_func
                            return func.__get__(self, type(self))(*args2, **kwargs2) ...
▶ Local vars
D:\Program Files\PythonEnv\foodgreen\lib\site-packages\django\utils\decorators.py in inner
                                return func(*args, **kwargs) ...
▶ Local vars
D:\Program Files\PythonEnv\foodgreen\lib\site-packages\django\contrib\admin\options.py in changeform_view
                                change_message = self.construct_change_message(request, form, formsets) ...
▶ Local vars
D:\Program Files\PythonEnv\foodgreen\lib\site-packages\django\contrib\admin\options.py in construct_change_message
                    if form.changed_data: ...
▶ Local vars
D:\Program Files\PythonEnv\foodgreen\lib\site-packages\django\forms\forms.py in changed_data
                            if field.has_changed(initial_value, data_value): ...
▶ Local vars
D:\Program Files\PythonEnv\foodgreen\lib\site-packages\django\contrib\postgres\forms\hstore.py in has_changed
                    initial_value = self.to_python(initial) ...
▶ Local vars
D:\Program Files\PythonEnv\foodgreen\lib\site-packages\django\contrib\postgres\forms\hstore.py in to_python
                        value = json.loads(value) ...
▶ Local vars
C:\Python27\Lib\json\__init__.py in loads
                    return _default_decoder.decode(s) ...
▶ Local vars
C:\Python27\Lib\json\decoder.py in decode
                    obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ...
▶ Local vars
Request information

GET
No GET data
POST
Variable	Value
rate_count	
u'0'
code	
u'P000002'
foodimage_set-1-description	
u''
weight	
u'{"1.5": "1.1kg - 2kg", "0.5": "0.5kg - 1kg"}'
photo	
u''
rate	
u'4'
foodimage_set-0-description	
u''
foodimage_set-0-ordering	
u'0'
foodimage_set-0-food	
u'2'
foodimage_set-TOTAL_FORMS	
u'2'
unit	
u'con,kg'
description	
u'<div><div>Chi ti\u1ebft s\u1ea3n ph\u1ea9m Th\u1ecbt g\xe0 s\u1ea1ch</div><div><br></div><div>Ch\xfang ta th\u01b0\u1eddng nghe \u0111\u1ebfn rau qu\u1ea3 s\u1ea1ch, tr\u1ee9ng s\u1ea1ch..., nh\u01b0ng l\u1ea1i qu\xean r\u1eb1ng th\u1ecbt c\u0169ng c\xf3 th\u1ec3 \u0111\u01b0\u1ee3c l\u1ea5y t\u1eeb gia s\xfac nu\xf4i b\u1eb1ng th\u1ee9c \u0103n th\u1ef1c v\u1eadt \u201csinh h\u1ecdc\u201d... v\u1edbi m\u1ed9t quy tr\xecnh ph\xf9 h\u1ee3p v\u1edbi t\u1ef1 nhi\xean. Th\u1ecbt \u0111\xf3 ch\xednh l\xe0 \u201cth\u1ecbt s\u1ea1ch\u201d. N\xf3i \u0111\u01a1n gi\u1ea3n h\u01a1n, th\u1ecbt s\u1ea1ch l\xe0 s\u1ea3n ph\u1ea9m kh\xf4ng c\xf3 thu\u1ed1c, kh\xe1ng sinh, hormon, ch\u1ea5t k\xedch th\xedch t\u0103ng tr\u01b0\u1edfng... Gia s\xfac ch\u1ec9 \u0103n c\u1ecf r\u01a1m hay ng\u0169 c\u1ed1c c\xf3 ch\u1ee9ng nh\u1eadn sinh h\u1ecdc kh\xf4ng thu\u1ed1c tr\u1eeb s\xe2u hay ph\xe2n b\xf3n h\xf3a h\u1ecdc.</div><div><br></div><div>C\xf3 th\u1ec3 th\u1ea5y, th\u1ecbt s\u1ea1ch t\u1eeb qu\xe1 tr\xecnh \u201cch\u0103n nu\xf4i s\u1ea1ch\u201d c\xf3 l\u1ee3i v\xe0 an to\xe0n h\u01a1n cho s\u1ee9c kh\u1ecfe. M\u1ed9t l\u1ee3i \xedch kh\xe1c khi ti\xeau thu th\u1ecbt s\u1ea1ch - l\xe0 g\xf3p ph\u1ea7n v\xe0o s\u1ef1 ph\xe1t tri\u1ec3n c\u1ee7a m\u1ed9t n\u1ec1n n\xf4ng nghi\u1ec7p b\u1ec1n v\u1eefng, t\xf4n tr\u1ecdng m\xf4i tr\u01b0\u1eddng v\xe0 ph\xf9 h\u1ee3p v\u1edbi thi\xean nhi\xean.</div><div><br></div><div><br></div><div>\u0110\u1eb6C \u0110I\u1ec2M N\u1ed4I B\u1eacT</div><div><br></div><div>Ch\u1ee9ng nh\u1eadn ti\xeau chu\u1ea9n s\u1ea3n ph\u1ea9m 246 / 2009 / YT\u0110N \u2013 CNTC.</div><div><br></div><div>S\u1ea3n ph\u1ea9m ch\u1ebf bi\u1ebfn ra \u0111\u01b0\u1ee3c b\u1ea3o qu\u1ea3n trong kho l\u1ea1nh tr\u01b0\u1edbc khi v\u1eadn chuy\u1ec3n t\u1edbi kh\xe1ch h\xe0ng. &nbsp;V\u1edbi h\u1ec7 th\u1ed1ng xe l\u1ea1nh chuy\xean d\u1ee5ng b\u1ea3o qu\u1ea3n s\u1ea3n ph\u1ea9m t\u1edbi kh\xe1ch h\xe0ng</div><div><br></div><div>S\u1ea3n ph\u1ea9m th\u1ecbt g\xe0 s\u1ea1ch \u0111\u01b0\u1ee3c cung c\u1ea5p cho th\u1ecb tr\u01b0\u1eddng TP.HCM v\xe0 c\xe1c h\u1ec7 th\u1ed1ng si\xeau th\u1ecb tr\xean to\xe0n qu\u1ed1c.</div><div><br></div><div>\u0110\u1ebfn v\u1edbi Green Healthy c\xe1c b\u1ea1n c\xf3 th\u1ec3 mua \u0111\u01b0\u1ee3c r\u1ea5t nhi\u1ec1u c\xe1c s\u1ea3n ph\u1ea9m t\u1eeb th\u1ecbt g\xe0 c\xf4ng nghi\u1ec7p th\u1ea3 v\u01b0\u1eddn nh\u01b0: g\xe0 nguy\xean con, c\xe1nh g\xe0 nguy\xean, g\xe0 g\xf3c t\u01b0, \u0111\xf9i g\xe0 t\u1ecfi, \u1ee9c g\xe0 phi l\xea, l\xf2ng g\xe0 v\u1edbi gi\xe1 c\u1ea3 r\u1ea5t kinh t\u1ebf.</div><div><br></div><div>B\u1eadt m\xed l\xe0 khi ch\u1ebf bi\u1ebfn m\xf3n \u0103n t\u1eeb th\u1ecbt g\xe0, s\u1eed d\u1ee5ng c\xf9ng lo\u1ea1i th\u1ecbt nh\u01b0 c\xe1nh, \u0111\xf9i s\u1ebd khi\u1ebfn cho m\xf3n \u0103n tr\xf4ng h\u1ea5p d\u1eabn nhi\u1ec1u so v\u1edbi khi ch\xfang ta d\xf9ng nhi\u1ec1u ph\u1ea7n th\u1ecbt tr\xean m\u1ed9t con g\xe0.&nbsp;</div></div><div><br></div>'
category	
u'9'
view_count	
u'0'
knowledge	
u'Th\u1ecbt g\xe0 ch\u1ee9a nhi\u1ec1u&nbsp;\u0111\u1ea1m r\u1ea5t t\u1ed1t cho con ng\u01b0\u1eddi&nbsp;'
foodimage_set-1-id	
u''
storage	
u'Tr\u1eed l\u1ea1nh nguy\xean con.'
unit_price	
u'kg'
foodimage_set-1-image	
u''
csrfmiddlewaretoken	
u'kulTr1CR3Co8Y5Q0HHgsgdeaY7jdtFPe'
foodimage_set-0-image	
u''
foodimage_set-MAX_NUM_FORMS	
u'1000'
files	
u''
foodimage_set-1-title	
u''
_save	
u'Save'
price	
u'200000'
discount	
u'0'
active	
u'on'
foodimage_set-0-id	
u'1'
foodimage_set-__prefix__-id	
u''
foodimage_set-1-food	
u'2'
foodimage_set-1-ordering	
u'0'
name	
u'G\xe0 ta'
foodimage_set-__prefix__-image	
u''
price_markup	
u'0'
made_in	
u'<p style="transition: all 0.25s ease-in-out; margin: 0px; padding-top: 0px; padding-bottom: 0px; border: 0px; font-family: inherit; font-size: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; line-height: 23px;">Ngu\u1ed3n g\u1ed1c h\xe0ng: ch\u1ee3 \u0111\u1ea7u m\u1ed1i</p>'
foodimage_set-__prefix__-description	
u''
foodimage_set-MIN_NUM_FORMS	
u'0'
foodimage_set-__prefix__-title	
u''
foodimage_set-0-title	
u'S\u01a1 ch\u1ebf nguy\xean tr\u1ea1ng'
foodimage_set-__prefix__-ordering	
u'0'
foodimage_set-__prefix__-food	
u'2'
foodimage_set-INITIAL_FORMS	
u'1'

Change History (8)

comment:1 by Tim Graham, 9 years ago

Description: modified (diff)

comment:2 by Tim Graham, 9 years ago

Resolution: invalid
Status: newclosed
Summary: Unable save model in admin with HStoreField fieldUnable save model in admin with HStoreField field after manually saving invalid data

JSON requires double quotes for strings. I don't think we should strive to make things work if you manually save invalid data to the field.

comment:3 by Marlon, 9 years ago

Resolution: invalid
Status: closednew

I see this same error in the Admin even when entering valid data with double quotes, using Django 1.8.3 and Python 2.7.10:

patient = Patient.objects.create(user=user, gender="male", mobile_telecom="555-555-4456", emergency_contacts={"urgent_care":"555-555-5555"}, utc_offset=0, birthdate=datetime.datetime.now())

Returning emergency_contacts from the shell results in single-quoted data:

In [23]: patient.emergency_contacts
Out[23]: {'urgent_care': '555-555-5555'}

From the traceback, it seems django's changed_data in forms.py is transforming the double-quoted data to single-quoted data:

/lib/python2.7/site-packages/django/forms/forms.py in changed_data

                        try:

                            initial_value = field.to_python(hidden_widget.value_from_datadict(

                                self.data, self.files, initial_prefixed_name))

                        except ValidationError:

                            # Always assume data has changed if validation fails.

                            self._changed_data.append(name)

                            continue

                                if field.has_changed(initial_value, data_value):

     ...

                        self._changed_data.append(name)

            return self._changed_data

        @property

        def media(self):

            """

▼ Local vars
Variable 	Value
data_value 	

u'{"urgent_care": "555-555-5555"}'

name 	

'emergency_contacts'

initial_value 	

{u'urgent_care': u'555-555-5555'}

self 	

<PatientForm bound=True, valid=True, fields=(user;gender;birthdate;mobile_telecom;emergency_contacts;utc_offset;hipaa_authorization_timestamp)>

field 	

<django.contrib.postgres.forms.hstore.HStoreField object at 0x105d54450>

prefixed_name 	

'emergency_contacts'

This is using the following model:

class Patient(TimeStampedModel):
    id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
    user = models.OneToOneField(settings.AUTH_USER_MODEL)
    gender = models.CharField(max_length=10, choices=GENDER_CHOICES)
    birthdate = models.DateField()
    mobile_telecom = PhoneNumberField(unique=True)
    emergency_contacts = HStoreField(default={"urgent_care": ""})
    utc_offset = models.IntegerField(default=0)
    hipaa_authorization_timestamp = models.DateTimeField(null=True, blank=True)
Version 2, edited 9 years ago by Marlon (previous) (next) (diff)

comment:4 by Tim Graham, 9 years ago

Owner: set to Tim Graham
Severity: NormalRelease blocker
Status: newassigned
Triage Stage: UnreviewedAccepted

comment:5 by Tim Graham, 9 years ago

Has patch: set

comment:6 by Simon Charette, 9 years ago

Triage Stage: AcceptedReady for checkin

comment:7 by Tim Graham <timograham@…>, 9 years ago

Resolution: fixed
Status: assignedclosed

In a7b7f27c:

Fixed #25233 -- Fixed HStoreField.has_changed() handling of initial values.

Thanks Simon Charette for review.

comment:8 by Tim Graham <timograham@…>, 9 years ago

In ad2ac53:

[1.8.x] Fixed #25233 -- Fixed HStoreField.has_changed() handling of initial values.

Thanks Simon Charette for review.

Backport of a7b7f27c05244d69a11545261eb3bbd73791b3d2 from master

Note: See TracTickets for help on using tickets.
Back to Top