Opened 4 years ago

Closed 11 months ago

#27391 closed New feature (fixed)

Support unittest.TestCase.debug() method.

Reported by: Pavel Savchenko Owned by: nobody
Component: Testing framework Version: master
Severity: Normal Keywords: unittest SimpleTestCase debug
Cc: me@… Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no



The documentation for unittest.TestCase.debug states that invoking it would:

Run the test without collecting the result. This allows exceptions raised by the test to be propagated to the caller, and can be used to support running tests under a debugger.

In essence, debug() invokes the test method same as does, without the added functionality of catching exceptions and related cruft.

Side effects

A recent change in pytest (see references below) brought to our attention the fact that Django does not implement a debug method, and performs important stuff in TestCase.__call__ (besides calling run()), such as calling _pre_startup and _post_teardown methods.

Thus, since debug() is unimplemented, running debug on a django TestCase (for example for debugging purposes), results unintended behavior, such as certain methods not being called. Most importantly _fixture_callback and _fixture_teardown which also perform database rollback (or call flush).


Change History (17)

comment:1 Changed 4 years ago by Tim Graham

Component: UncategorizedTesting framework
Type: UncategorizedNew feature

Would this have benefit outside of the usage of third-party test runners such as pytest? If so, what does it look like?

comment:2 Changed 4 years ago by Pavel Savchenko

Well, not that I know of yet, but basically, any test runner that allows unittests may assume "debug" can be used for exception debugging; and right now this cannot be done without issues when using Django's SimpleTestCase derivatives.

Regardless, the patch should be fairly simple 10 line method and I can submit that. It's invoking the test that I'm having trouble with, how to invoke "debug" as part of the regular test suite, maybe we can use call_command...

Last edited 4 years ago by Pavel Savchenko (previous) (diff)

comment:3 Changed 4 years ago by Tim Graham

Triage Stage: UnreviewedAccepted

comment:4 Changed 4 years ago by Pavel Savchenko

As for other test runners, here's now nose2 invokes a debugger (IIRC pytest does something similar):

This means that the post_mortem is invoked only after the exception was already caught in SimpleTestCase.__call__ and, for example, the database was already rolled back. The test frameworks simple have no real way to examine the state of the application at the time of the error (but only "after", when it may be too late).

comment:5 Changed 4 years ago by Pavel Savchenko

Has patch: set

Added test case (demonstrating usage expectations) and resolution in PR 7436

comment:6 Changed 4 years ago by Adam (Chainz) Johnson

Cc: me@… added

comment:7 Changed 4 years ago by Tim Graham

Patch needs improvement: set

I think the test could be a bit cleaner. I sent some edits to the PR, though I broke some things too.

comment:8 Changed 3 years ago by Tim Graham

Patch needs improvement: unset

comment:9 Changed 3 years ago by Tim Martin

Patch needs improvement: set

comment:10 Changed 2 years ago by Asif Saifuddin Auvi

Needs documentation: set
Version: 1.10master

comment:11 in reply to:  10 Changed 2 years ago by Pavel Savchenko

Patch needs improvement: unset

Replying to Asif Saifuddin Auvi:

Patch was improved as per Tim Martin's notes in commit 7ec5fae. I added a sketch for release note mention in this comment.

Is there other documentation you had in mind?

comment:12 Changed 2 years ago by Tim Graham

Needs documentation: unset

comment:13 Changed 2 years ago by Tim Graham

Patch needs improvement: set

comment:14 Changed 12 months ago by felixxm

Patch needs improvement: unset

comment:15 Changed 11 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In dc8cd2fe:

Refs #27391 -- Added more tests for SimpleTestCase.

comment:16 Changed 11 months ago by felixxm

Summary: Support unittest.TestCase.debug() methodSupport unittest.TestCase.debug() method.
Triage Stage: AcceptedReady for checkin

comment:17 Changed 11 months ago by Mariusz Felisiak <felisiak.mariusz@…>

Resolution: fixed
Status: newclosed

In 1711c509:

Fixed #27391 -- Implemented SimpleTestCase.debug().

debug() should bubbled up exceptions if occurring in test, but behave
the same as run() when no exceptions occurred.

Note: See TracTickets for help on using tickets.
Back to Top