Opened 12 years ago

Closed 12 years ago

Last modified 12 years ago

#18340 closed Bug (fixed)

formtools.utils.form_hmac is not consistent with unicode input

Reported by: Claude Paroz Owned by: nobody
Component: contrib.formtools Version: dev
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


This problem can only be reproduced with cPickle. When feeding the django.contrib.formtools.utils.form_hmac function with Unicode values, stripping the leading/ending spaces produces different pickled results than the unstripped identical values. Here is an output of the two form_hmac passes in the FormHmacTests.test_textfield_hash when you convert form values to Unicode:

(Pdb) p data, pickled
([('name', u'joe'), ('bio', u'Nothing notable.')], '\x80\x02]q\x01(U\x04nameq\x02X\x03\x00\x00\x00joeq\x03\x86q\x04U\x03bioq\x05X\x10\x00\x00\x00Nothing notable.q\x06\x86q\x07e.')

(Pdb) p data, pickled
([('name', u'joe'), ('bio', u'Nothing notable.')], '\x80\x02]q\x01(U\x04nameq\x02X\x03\x00\x00\x00joe\x86q\x03U\x03bioq\x04X\x10\x00\x00\x00Nothing notable.q\x05\x86q\x06e.')

Possible workarounds:

  • use the (slower) pickle module only, as I don't think the performance penalty will be noticeable in this part of the code
  • encode all unicode values to 'utf-8' (smart_str) before feeding them to pickle.dumps
  • feed repr(data) to pickle.dumps

Attachments (1)

18340.diff (1.4 KB ) - added by Claude Paroz 12 years ago.
Do not use cPickle

Download all attachments as: .zip

Change History (6)

by Claude Paroz, 12 years ago

Attachment: 18340.diff added

Do not use cPickle

comment:1 by Claude Paroz, 12 years ago

Has patch: set

comment:2 by Claude Paroz <claude@…>, 12 years ago

Resolution: fixed
Status: newclosed

In [1a66f53f9413583d1da266356bb35e5f82868641]:

Fixed #18340 -- Fixed formtools form_hmac with Unicode input

Using cPickle, two apparently identical Unicode strings could
generate different pickled results depending on previous operations
on those strings.

comment:3 by Claude Paroz <claude@…>, 12 years ago

In [078ea51b1c7504e3f941bc1d96b0499ace29bdb0]:

Fixed test failures after commit 1a66f53. Refs #18340

comment:4 by anonymous, 12 years ago

It's worth noting that while pickle currently seems to be idempotent, it's not guaranteed to anywhere in the spec. Relying on pickling python objects might not be the ideal solution here.

comment:5 by Claude Paroz <claude@…>, 12 years ago

In [b109ff8062f4bb225181ec462d69c9dd79339567]:

Replaced pickle by json in form_hmac calculation

Refs #18340

Note: See TracTickets for help on using tickets.
Back to Top