Opened 4 years ago

Closed 4 years ago

#15805 closed Bug (fixed)

assertFieldOutput should not use assertRaisesRegexp

Reported by: julien Owned by: nobody
Component: Testing framework Version: master
Severity: Release blocker Keywords:
Cc: lawgon@… Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX:

Description

The fact that assertFieldOutput uses assertRaisesRegexp causes some weird and inconsistent behaviours, which I describe in more detail in #14608 and in http://groups.google.com/group/django-developers/browse_thread/thread/96da81267aadade0

Not only does it make no sense to use regular expressions here, but also pretty much all the tests in regressiontests.forms.localflavor using assertFieldOutput are currently passing without being absolutely sure that the expected error messages are correct (To verify that, try modifying any error message in those tests -- the tests will still pass).

See attached patch fixing the behaviour and also straightening some tests that were incorrectly passing before.

Marking as Release Blocker since, in effect, the test suite is currently broken because of this behaviour.

Attachments (5)

15805.assertFieldOutput.diff (5.1 KB) - added by julien 4 years ago.
15805.assertFieldOutput.2.diff (7.1 KB) - added by julien 4 years ago.
Same patch with regression tests for assertFieldOutput itself
15805.assertFieldOutput.3.diff (8.0 KB) - added by julien 4 years ago.
Updated patch to also fix broken tests for some recently added localflavors (HR and AU)
15805.assertFieldOutput.4.diff (8.5 KB) - added by julien 4 years ago.
Had forgotten to add AssertFieldOutputTests to the list of tests to run
15805.assertFieldOutput.5.diff (8.6 KB) - added by julien 4 years ago.
Patch updated to latest trunk

Download all attachments as: .zip

Change History (12)

Changed 4 years ago by julien

comment:1 Changed 4 years ago by julien

  • Needs tests set

This actually needs some regression tests for assertFieldOutput itself. I'm working on it.

Changed 4 years ago by julien

Same patch with regression tests for assertFieldOutput itself

comment:2 Changed 4 years ago by julien

  • Needs tests unset

I've added the regression tests. I think the assertFieldOutput is actually quite useful. Also it's quite generic and not really local flavor specific. So I'd suggest we eventually move it to the django.utils.unittest.TestCase class and document it so that the world knows about it. I'll create a new ticket for that if/when this patch gets checked in.

comment:3 Changed 4 years ago by lawgon

  • Cc lawgon@… added

comment:4 Changed 4 years ago by lrekucki

  • Easy pickings unset
  • Triage Stage changed from Unreviewed to Accepted

comment:5 Changed 4 years ago by julien

For the record, I've created #15838 to suggest moving assertFieldOutput to the general TestCase.

Changed 4 years ago by julien

Updated patch to also fix broken tests for some recently added localflavors (HR and AU)

Changed 4 years ago by julien

Had forgotten to add AssertFieldOutputTests to the list of tests to run

comment:6 Changed 4 years ago by claudep

  • Triage Stage changed from Accepted to Ready for checkin
  • Version changed from 1.2 to SVN

In the future, it would also be cool if the failing message would expose the exact value which is failing, instead of simply outputting the raw ValidationError. But this is already a nice improvement.

Changed 4 years ago by julien

Patch updated to latest trunk

comment:7 Changed 4 years ago by lukeplant

  • Resolution set to fixed
  • Status changed from new to closed

In [16303]:

Fixed #15805 - assertFieldOutput should not use assertRaisesRegexp

Thanks to julien for the report and patch.

Note: See TracTickets for help on using tickets.
Back to Top