Opened 8 years ago
Closed 5 years ago
#27391 closed New feature (fixed)
Support unittest.TestCase.debug() method.
Reported by: | Pavel Savchenko | Owned by: | nobody |
---|---|---|---|
Component: | Testing framework | Version: | dev |
Severity: | Normal | Keywords: | unittest SimpleTestCase debug |
Cc: | me@… | Triage Stage: | Ready for checkin |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
Introduction
The documentation for unittest.TestCase.debug states that invoking it would:
Run the test without collecting the result. This allows exceptions raised by the test to be propagated to the caller, and can be used to support running tests under a debugger.
In essence, debug()
invokes the test method same as TestCase.run() does, without the added functionality of catching exceptions and related cruft.
Side effects
A recent change in pytest (see references below) brought to our attention the fact that Django does not implement a debug
method, and performs important stuff in TestCase.__call__
(besides calling run()
), such as calling _pre_startup
and _post_teardown
methods.
Thus, since debug() is unimplemented, running debug on a django TestCase (for example for debugging purposes), results unintended behavior, such as certain methods not being called. Most importantly _fixture_callback
and _fixture_teardown
which also perform database rollback (or call flush
).
References:
https://github.com/pytest-dev/pytest/pull/1890
https://github.com/pytest-dev/pytest/issues/1932
https://github.com/pytest-dev/pytest-django/pull/406
https://github.com/python/cpython/blob/master/Lib/unittest/case.py#L651-L658
Change History (17)
comment:1 by , 8 years ago
Component: | Uncategorized → Testing framework |
---|---|
Type: | Uncategorized → New feature |
comment:2 by , 8 years ago
Well, not that I know of yet, but basically, any test runner that allows unittests may assume "debug" can be used for exception debugging; and right now this cannot be done without issues when using Django's SimpleTestCase derivatives.
Regardless, the patch should be fairly simple 10 line method and I can submit that. It's invoking the test that I'm having trouble with, how to invoke "debug" as part of the regular test suite...
comment:3 by , 8 years ago
Triage Stage: | Unreviewed → Accepted |
---|
comment:4 by , 8 years ago
As for other test runners, here's now nose2 invokes a debugger (IIRC pytest does something similar):
https://github.com/nose-devs/nose2/blob/master/nose2/plugins/debugger.py#L41-L60
This means that the post_mortem
is invoked only after the exception was already caught in SimpleTestCase.__call__
and, for example, the database was already rolled back. The test frameworks simple have no real way to examine the state of the application at the time of the error (but only "after", when it may be too late).
comment:5 by , 8 years ago
Has patch: | set |
---|
Added test case (demonstrating usage expectations) and resolution in PR 7436
comment:6 by , 8 years ago
Cc: | added |
---|
comment:7 by , 8 years ago
Patch needs improvement: | set |
---|
I think the test could be a bit cleaner. I sent some edits to the PR, though I broke some things too.
comment:8 by , 7 years ago
Patch needs improvement: | unset |
---|
comment:9 by , 7 years ago
Patch needs improvement: | set |
---|
follow-up: 11 comment:10 by , 6 years ago
Needs documentation: | set |
---|---|
Version: | 1.10 → master |
comment:11 by , 6 years ago
Patch needs improvement: | unset |
---|
Replying to Asif Saifuddin Auvi:
Patch was improved as per Tim Martin's notes in commit 7ec5fae. I added a sketch for release note mention in this comment.
Is there other documentation you had in mind?
comment:12 by , 6 years ago
Needs documentation: | unset |
---|
comment:13 by , 6 years ago
Patch needs improvement: | set |
---|
comment:14 by , 5 years ago
Patch needs improvement: | unset |
---|
comment:16 by , 5 years ago
Summary: | Support unittest.TestCase.debug() method → Support unittest.TestCase.debug() method. |
---|---|
Triage Stage: | Accepted → Ready for checkin |
Would this have benefit outside of the usage of third-party test runners such as
pytest
? If so, what does it look like?