Code

Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#18340 closed Bug (fixed)

formtools.utils.form_hmac is not consistent with unicode input

Reported by: claudep Owned by: nobody
Component: contrib.formtools Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

This problem can only be reproduced with cPickle. When feeding the django.contrib.formtools.utils.form_hmac function with Unicode values, stripping the leading/ending spaces produces different pickled results than the unstripped identical values. Here is an output of the two form_hmac passes in the FormHmacTests.test_textfield_hash when you convert form values to Unicode:

(Pdb) p data, pickled
([('name', u'joe'), ('bio', u'Nothing notable.')], '\x80\x02]q\x01(U\x04nameq\x02X\x03\x00\x00\x00joeq\x03\x86q\x04U\x03bioq\x05X\x10\x00\x00\x00Nothing notable.q\x06\x86q\x07e.')

(Pdb) p data, pickled
([('name', u'joe'), ('bio', u'Nothing notable.')], '\x80\x02]q\x01(U\x04nameq\x02X\x03\x00\x00\x00joe\x86q\x03U\x03bioq\x04X\x10\x00\x00\x00Nothing notable.q\x05\x86q\x06e.')

Possible workarounds:

  • use the (slower) pickle module only, as I don't think the performance penalty will be noticeable in this part of the code
  • encode all unicode values to 'utf-8' (smart_str) before feeding them to pickle.dumps
  • feed repr(data) to pickle.dumps

Attachments (1)

18340.diff (1.4 KB) - added by claudep 2 years ago.
Do not use cPickle

Download all attachments as: .zip

Change History (6)

Changed 2 years ago by claudep

Do not use cPickle

comment:1 Changed 2 years ago by claudep

  • Has patch set

comment:2 Changed 2 years ago by Claude Paroz <claude@…>

  • Resolution set to fixed
  • Status changed from new to closed

In [1a66f53f9413583d1da266356bb35e5f82868641]:

Fixed #18340 -- Fixed formtools form_hmac with Unicode input

Using cPickle, two apparently identical Unicode strings could
generate different pickled results depending on previous operations
on those strings.

comment:3 Changed 2 years ago by Claude Paroz <claude@…>

In [078ea51b1c7504e3f941bc1d96b0499ace29bdb0]:

Fixed test failures after commit 1a66f53. Refs #18340

comment:4 Changed 2 years ago by anonymous

It's worth noting that while pickle currently seems to be idempotent, it's not guaranteed to anywhere in the spec. Relying on pickling python objects might not be the ideal solution here.

comment:5 Changed 2 years ago by Claude Paroz <claude@…>

In [b109ff8062f4bb225181ec462d69c9dd79339567]:

Replaced pickle by json in form_hmac calculation

Refs #18340

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.