Opened 15 years ago

Closed 12 years ago

#11603 closed New feature (fixed)

Add an assertFormSetError function to django.test.TestCase

Reported by: martin_speleo Owned by: martin_speleo
Component: Testing framework Version: dev
Severity: Normal Keywords: formset
Cc: martin.speleo@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

As well having the function assertFormError in TransactionTestCase see (source:django/trunk/django/test/testcases.py), a similar function for checking for errors in formsets would be usefull. It would look for

  • field errors
  • non-field errors
  • non-form errors

This is a similar request to ticket:11241 but may be neater and have a larger scope. A possible solution is this modification of assertFormError:

    def assertFormSetError(self, response, formset, index, field, errors):
        """
        Asserts that a formset used to render the response has a specific error.
        """
        # Put context(s) into a list to simplify processing.
        contexts = to_list(response.context)
        if not contexts:
            self.fail('Response did not use any contexts to render the response')

        # Put error(s) into a list to simplify processing.
        errors = to_list(errors)

        # Search all contexts for the error.
        found_formset = False
        for i,context in enumerate(contexts):
            if formset not in context:
                continue
            found_formset = True
            for err in errors:
                if field:
                    if field in context[formset].forms[index].errors:
                        field_errors = context[formset].forms[index].errors[field]
                        self.failUnless(err in field_errors,
                                "The field '%s' on formset '%s' [%s] in"
                                " context %d does not contain the"
                                " error '%s' (actual errors: %s)" %
                                        (field, formset, index, i, err,
                                        repr(field_errors)))
                    elif field in context[formset].forms[index].fields:
                        self.fail("The field '%s' on formset '%s' [%s] in context %d"
                                  " contains no errors" % (field, form, index, i))
                    else:
                        self.fail("The formset '%s' [%s] in context %d does not"
                                  " contain the field '%s'" %
                                        (formset, index, i, field))
                elif index is not None:
                    non_field_errors = context[formset].forms[index].non_field_errors()
                    self.failUnless(err in non_field_errors,
                        "The formset '%s' [%s] in context %d does not contain the"
                        " non-field error '%s' (actual errors: %s)" %
                        (formset, index, i, err, non_field_errors))
                else:
                    print dir(context[formset])
                    non_form_errors = context[formset].non_form_errors()
                    self.failUnless(err in non_form_errors,
                        "The formset '%s' in context %d does not contain the"
                        " non-form error '%s' (actual errors: %s)" %
                        (formset, i, err, non_form_errors))
        if not found_formset:
            self.fail("The formset '%s' was not used to render the response" % formset)

Let me know if this is a sensible approach, and if so I could work on diffs and unittests for it.

Attachments (3)

assertFormsetError.diff (20.6 KB ) - added by martin_speleo 14 years ago.
assertformseterror.diff (20.8 KB ) - added by martin_speleo 13 years ago.
Fixed failing unittests, changed form parameter to form_index
assertformseterror.diff.patch (20.4 KB ) - added by martin_speleo 13 years ago.
Maintance update of patch, and removal of entry in release notes for 1.4

Download all attachments as: .zip

Change History (20)

comment:1 by Alex Gaynor, 15 years ago

Triage Stage: UnreviewedAccepted

comment:2 by martin_speleo, 15 years ago

Cc: sethtrain@… removed
Needs documentation: set
Needs tests: set
Owner: changed from nobody to martin_speleo

comment:3 by martin_speleo, 15 years ago

Has patch: set
Needs documentation: unset
Needs tests: unset
Version: SVN

My first attempt at contibuting to Django, I think the patch should be ready to be checked in.

It could possibly do with some unittesting on checking for more than one error at once. assertFormError also does not have unittests for dealing with multiple errors, along with a few other little bits and pieces. But if I understand correctly, exceptions are normally used to raise errors in forms/formsets so mulitple errors will be uncommon.

I have not tested that my changes to the documentation do not break everything, as I am running Vista and the make file for the documentation used a -p command on mkdir which did not appear to be understood. If this is a problem let me know and I will try to make the documenetation again.

comment:4 by Julien Phalip, 14 years ago

Severity: Normal
Type: New feature

comment:5 by Julien Phalip, 14 years ago

Easy pickings: unset
Patch needs improvement: set

Thanks for your patch, it looks pretty good. It would need to be updated to apply to trunk though. Also, the tests could be made more elegant by using assertRaises and the with statement (http://docs.python.org/library/unittest.html#unittest.TestCase.assertRaises) instead of all those try/except/else's. By the way, you can import the with statement in Python 2.5 by doing: from __future__ import with_statement

comment:6 by martin_speleo, 14 years ago

Patch needs improvement: unset
UI/UX: unset

I have updated the patch to include support for the optional msq_prefix keyword argument in trunk. The tests now use assertRaises and the with statement. The documentation states that it is a new feature in SVN, let me know if I need to change this to 1.4, and add to the 1.4 release notes.

by martin_speleo, 14 years ago

Attachment: assertFormsetError.diff added

comment:7 by martin_speleo, 14 years ago

milestone: 1.4

Documentation changed assuming that this will be added to version 1.4, along with a minor feature entry in the v1.4 release notes.

comment:8 by Russell Keith-Magee, 14 years ago

Triage Stage: AcceptedReady for checkin

comment:9 by Russell Keith-Magee, 13 years ago

Patch needs improvement: set
Triage Stage: Ready for checkinAccepted

I just tried to apply this patch, and it failed 4 of its own tests.

On a closer examination, I'm also inclined to suggest that the form argument should be named form_index; there are other assertions that have a form argument that take an actual form instance, so form_index makes it clear that you're asking for an integer, not a form.

in reply to:  9 comment:10 by anonymous, 13 years ago

Replying to russellm:

I just tried to apply this patch, and it failed 4 of its own tests.

I am unable to reproduce these test failures, could you send me the errors that you get, and any relevant version information.

Thanks,
Martin

comment:11 by martin.speleo@…, 13 years ago

Cc: martin.speleo@… added

comment:12 by Russell Keith-Magee, 13 years ago

As requested in a private chat, here are the errors I see running against r16707, Python 2.6.1, under OSX Snow Leopard.

django)kronkite:tests rkm$ ./runtests.py --settings=sqlite test_client test_client_regress
Creating test database for alias 'default'...
Creating test database for alias 'other'...
.................................................FF...F..FF..................................................................
======================================================================
FAIL: test_no_nonfield_error (regressiontests.test_client_regress.models.AssertFormsetErrorTests)
An assertion is raised if the formsets non-field errors doesn't contain any errors.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rkm/projects/django/hg/tests/regressiontests/test_client_regress/models.py", line 606, in test_no_nonfield_error
    prefix + "The formset 'my_formset', form 1 in "
AssertionError: "True is not False : The formset 'my_formset', form 1 in context 0 does not contain any non-field errors." != "The formset 'my_formset', form 1 in context 0 does not contain any non-field errors."

======================================================================
FAIL: test_no_nonform_error (regressiontests.test_client_regress.models.AssertFormsetErrorTests)
An assertion is raised if the formsets non-form errors doesn't contain any errors.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rkm/projects/django/hg/tests/regressiontests/test_client_regress/models.py", line 647, in test_no_nonform_error
    "The formset 'my_formset' in context 0 "
AssertionError: "True is not False : The formset 'my_formset' in context 0 does not contain any non-form errors." != "The formset 'my_formset' in context 0 does not contain any non-form errors."

======================================================================
FAIL: test_unknown_error (regressiontests.test_client_regress.models.AssertFormsetErrorTests)
An assertion is raised if the field doesn't contain the specified error
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rkm/projects/django/hg/tests/regressiontests/test_client_regress/models.py", line 579, in test_unknown_error
    prefix + "The field 'email' "
AssertionError: "False is not True : The field 'email' on formset 'my_formset', form 0 in context 0 does not contain the error 'Some error.' (actual errors: [u'Enter a valid e-mail address.'])" != "The field 'email' on formset 'my_formset', form 0 in context 0 does not contain the error 'Some error.' (actual errors: [u'Enter a valid e-mail address.'])"

======================================================================
FAIL: test_unknown_nonfield_error (regressiontests.test_client_regress.models.AssertFormsetErrorTests)
An assertion is raised if the formsets non-field errors doesn't contain the provided error.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rkm/projects/django/hg/tests/regressiontests/test_client_regress/models.py", line 621, in test_unknown_nonfield_error
    prefix + "The formset 'my_formset', form 0 in "
AssertionError: "False is not True : The formset 'my_formset', form 0 in context 0 does not contain the non-field error 'Some error.' (actual errors: [u'Non-field error.'])" != "The formset 'my_formset', form 0 in context 0 does not contain the non-field error 'Some error.' (actual errors: [u'Non-field error.'])"

======================================================================
FAIL: test_unknown_nonform_error (regressiontests.test_client_regress.models.AssertFormsetErrorTests)
An assertion is raised if the formsets non-form errors doesn't contain the provided error.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/rkm/projects/django/hg/tests/regressiontests/test_client_regress/models.py", line 661, in test_unknown_nonform_error
    "The formset 'my_formset' in context 0 does not "
AssertionError: "False is not True : The formset 'my_formset' in context 0 does not contain the non-form error 'Some error.' (actual errors: [u'Forms in a set must have distinct E-mail addresses.'])" != "The formset 'my_formset' in context 0 does not contain the non-form error 'Some error.' (actual errors: [u'Forms in a set must have distinct E-mail addresses.'])"

----------------------------------------------------------------------
Ran 125 tests in 3.638s

FAILED (failures=5)

by martin_speleo, 13 years ago

Attachment: assertformseterror.diff added

Fixed failing unittests, changed form parameter to form_index

comment:13 by martin_speleo, 13 years ago

Patch needs improvement: unset

I have modified the code and documentation, as suggested, to take a form_index parameter rather than a form parameter.

I have resolved the failures of the test. The issue was the default value of the longMessage variable (Boolean) in the unittest module. I have now checked that the tests work using python 2.6 and python 2.7, with and without unittest2 available. The documentation building has also been checked.

For AssertFormsetError the error messages are slightly clearer with longMessage set to False, however I have not set this parameter in order to stay consistent with the other test functions eg. AssertFormError.

comment:14 by Jacob, 13 years ago

milestone: 1.4

Milestone 1.4 deleted

by martin_speleo, 13 years ago

Maintance update of patch, and removal of entry in release notes for 1.4

comment:15 by Tim Graham, 12 years ago

I've used this in a project with good success. I've updated the patch to apply cleanly to trunk and submitted a pull request.

https://github.com/django/django/pull/708

comment:16 by Tim Graham, 12 years ago

I've updated the pull request above to ensure the tests pass on Python 3 and think this is RFC, but would appreciate if another core dev could take a quick look and +1 since my contributions have been limited to documentation.

comment:17 by Tim Graham <timograham@…>, 12 years ago

Resolution: fixed
Status: newclosed

In d194714c0a707773bd470bffb3d67a60e40bb787:

Fixed #11603 - Added django.test.SimpleTestCase.assertFormsetError

Thank-you Martin Green for the patch.

Note: See TracTickets for help on using tickets.
Back to Top