Opened 8 years ago

Closed 8 years ago

#19020 closed Cleanup/optimization (fixed)

Some contrib.formtools tests fail when Python hash value randomization is enabled

Reported by: Matt McDonald Owned by: nobody
Component: contrib.formtools Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no


The following tests in the contrib formtools application can fail when hash value randomization is enabled in the Python interpreter (2.7+). As written, these test cases currently rely on the implicit order of items in a dictionary, which is unsupported.

This test attempts to perform a regex to extract name/value pairs from an HTML string representing a list of <input> fields. The order of the attributes in the HTML is dependent on the sort order of the dictionary that originally represented them. Thus, a regex that expects the 'name' attribute to always directly precede the 'value' attribute will fail.
This test attempts to compare the result of JSON serialized dict to a literal string representation of that expected JSON. Since the order of the elements cannot be predicted, this comparison is prone to failure.

(For more background on hash randomization:

Here's an example command line that produces the failures (note -R flag to enable hash randomization):

$ python -R ./ --settings test_sqlite \
  formtools.TestCookieStorage.test_reset_cookie \
Creating test database for alias 'default'...
Creating test database for alias 'other'...
ERROR: test_9473 (django.contrib.formtools.tests.WizardTests)
Traceback (most recent call last):
  File "/tmp/django/django/contrib/formtools/tests/", line 442, in test_9473
    response = self.check_wizard_step(response, step_no)
  File "/tmp/django/django/contrib/formtools/tests/", line 437, in check_wizard_step
    return'/wizard2/', data)
  File "/tmp/django/django/test/", line 429, in post
    response = super(Client, self).post(path, data=data, content_type=content_type, **extra)
  File "/tmp/django/django/test/", line 263, in post
    return self.request(**r)
  File "/tmp/django/django/test/", line 390, in request
  File "/tmp/django/django/core/handlers/", line 115, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/tmp/django/django/utils/", line 25, in _wrapper
    return bound_func(*args, **kwargs)
  File "/tmp/django/django/utils/", line 91, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/tmp/django/django/utils/", line 21, in bound_func
    return func(self, *args2, **kwargs2)
  File "/tmp/django/django/contrib/formtools/wizard/", line 101, in __call__
    request, f):
  File "/tmp/django/django/contrib/formtools/wizard/", line 62, in _check_security_hash
    expected = self.security_hash(request, form)
  File "/tmp/django/django/contrib/formtools/wizard/", line 177, in security_hash
    return form_hmac(form)
  File "/tmp/django/django/contrib/formtools/", line 21, in form_hmac
    value = bf.field.clean( or ''
  File "/tmp/django/django/forms/", line 155, in clean
  File "/tmp/django/django/forms/", line 127, in validate
    raise ValidationError(self.error_messages['required'])
ValidationError: [u'This field is required.']

FAIL: test_reset_cookie (django.contrib.formtools.tests.wizard.cookiestorage.TestCookieStorage)
Traceback (most recent call last):
  File "/tmp/django/django/contrib/formtools/tests/wizard/", line 44, in test_reset_cookie
    self.assertEqual(unsigned_cookie_data, '{"step_files":{},"step":null,"extra_data":{},"step_data":{}}')
AssertionError: u'{"step_files":{},"extra_data":{},"step_data":{},"step":null}' != '{"step_files":{},"step":null,"extra_data":{},"step_data":{}}'

Ran 2 tests in 0.015s

FAILED (failures=1, errors=1)
Destroying test database for alias 'default'...
Destroying test database for alias 'other'...

Attachments (1)

19020-patch1.diff (2.3 KB) - added by Matt McDonald 8 years ago.

Download all attachments as: .zip

Change History (3)

Changed 8 years ago by Matt McDonald

Attachment: 19020-patch1.diff added

comment:1 Changed 8 years ago by Claude Paroz

Triage Stage: UnreviewedAccepted

This is a part of what has already been reported in #17758. I will combine your patch with some code found on that ticket to fix this one.

comment:2 Changed 8 years ago by Claude Paroz <claude@…>

Resolution: fixed
Status: newclosed

In 1f84b042f1c3fab0f806de814d668917dd86236a:

Fixed #19020 -- Do not depend on dict order in formtools tests

Thanks metzen for the report and initial patch, and Łukasz Rekucki
for an inspirational patch on his randomhash_fixes branch.

Note: See TracTickets for help on using tickets.
Back to Top