Opened 2 years ago

Closed 2 years ago

#19020 closed Cleanup/optimization (fixed)

Some contrib.formtools tests fail when Python hash value randomization is enabled

Reported by: metzen Owned by: nobody
Component: contrib.formtools Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description

The following tests in the contrib formtools application can fail when hash value randomization is enabled in the Python interpreter (2.7+). As written, these test cases currently rely on the implicit order of items in a dictionary, which is unsupported.

django.contrib.formtools.tests.WizardTests.test_9473
This test attempts to perform a regex to extract name/value pairs from an HTML string representing a list of <input> fields. The order of the attributes in the HTML is dependent on the sort order of the dictionary that originally represented them. Thus, a regex that expects the 'name' attribute to always directly precede the 'value' attribute will fail.
django.contrib.formtools.tests.wizard.cookiestorage.TestCookieStorage.test_reset_cookie
This test attempts to compare the result of JSON serialized dict to a literal string representation of that expected JSON. Since the order of the elements cannot be predicted, this comparison is prone to failure.

(For more background on hash randomization: http://bugs.python.org/issue13703)

Here's an example command line that produces the failures (note -R flag to enable hash randomization):

$ python -R ./runtests.py --settings test_sqlite \
  formtools.TestCookieStorage.test_reset_cookie \
  formtools.WizardTests.test_9473
Creating test database for alias 'default'...
Creating test database for alias 'other'...
FE
======================================================================
ERROR: test_9473 (django.contrib.formtools.tests.WizardTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/django/django/contrib/formtools/tests/__init__.py", line 442, in test_9473
    response = self.check_wizard_step(response, step_no)
  File "/tmp/django/django/contrib/formtools/tests/__init__.py", line 437, in check_wizard_step
    return self.client.post('/wizard2/', data)
  File "/tmp/django/django/test/client.py", line 429, in post
    response = super(Client, self).post(path, data=data, content_type=content_type, **extra)
  File "/tmp/django/django/test/client.py", line 263, in post
    return self.request(**r)
  File "/tmp/django/django/test/client.py", line 390, in request
    six.reraise(*exc_info)
  File "/tmp/django/django/core/handlers/base.py", line 115, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/tmp/django/django/utils/decorators.py", line 25, in _wrapper
    return bound_func(*args, **kwargs)
  File "/tmp/django/django/utils/decorators.py", line 91, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/tmp/django/django/utils/decorators.py", line 21, in bound_func
    return func(self, *args2, **kwargs2)
  File "/tmp/django/django/contrib/formtools/wizard/legacy.py", line 101, in __call__
    request, f):
  File "/tmp/django/django/contrib/formtools/wizard/legacy.py", line 62, in _check_security_hash
    expected = self.security_hash(request, form)
  File "/tmp/django/django/contrib/formtools/wizard/legacy.py", line 177, in security_hash
    return form_hmac(form)
  File "/tmp/django/django/contrib/formtools/utils.py", line 21, in form_hmac
    value = bf.field.clean(bf.data) or ''
  File "/tmp/django/django/forms/fields.py", line 155, in clean
    self.validate(value)
  File "/tmp/django/django/forms/fields.py", line 127, in validate
    raise ValidationError(self.error_messages['required'])
ValidationError: [u'This field is required.']

======================================================================
FAIL: test_reset_cookie (django.contrib.formtools.tests.wizard.cookiestorage.TestCookieStorage)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/django/django/contrib/formtools/tests/wizard/cookiestorage.py", line 44, in test_reset_cookie
    self.assertEqual(unsigned_cookie_data, '{"step_files":{},"step":null,"extra_data":{},"step_data":{}}')
AssertionError: u'{"step_files":{},"extra_data":{},"step_data":{},"step":null}' != '{"step_files":{},"step":null,"extra_data":{},"step_data":{}}'

----------------------------------------------------------------------
Ran 2 tests in 0.015s

FAILED (failures=1, errors=1)
Destroying test database for alias 'default'...
Destroying test database for alias 'other'...

Attachments (1)

19020-patch1.diff (2.3 KB) - added by metzen 2 years ago.

Download all attachments as: .zip

Change History (3)

Changed 2 years ago by metzen

comment:1 Changed 2 years ago by claudep

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

This is a part of what has already been reported in #17758. I will combine your patch with some code found on that ticket to fix this one.

comment:2 Changed 2 years ago by Claude Paroz <claude@…>

  • Resolution set to fixed
  • Status changed from new to closed

In 1f84b042f1c3fab0f806de814d668917dd86236a:

Fixed #19020 -- Do not depend on dict order in formtools tests

Thanks metzen for the report and initial patch, and Łukasz Rekucki
for an inspirational patch on his randomhash_fixes branch.

Note: See TracTickets for help on using tickets.
Back to Top