Opened 2 years ago

Last modified 3 months ago

#27391 new New feature

Support unittest.TestCase.debug() method

Reported by: Pavel Savchenko Owned by: nobody
Component: Testing framework Version: master
Severity: Normal Keywords: unittest SimpleTestCase debug
Cc: me@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

Introduction

The documentation for unittest.TestCase.debug states that invoking it would:

Run the test without collecting the result. This allows exceptions raised by the test to be propagated to the caller, and can be used to support running tests under a debugger.

In essence, debug() invokes the test method same as TestCase.run() does, without the added functionality of catching exceptions and related cruft.

Side effects

A recent change in pytest (see references below) brought to our attention the fact that Django does not implement a debug method, and performs important stuff in TestCase.__call__ (besides calling run()), such as calling _pre_startup and _post_teardown methods.

Thus, since debug() is unimplemented, running debug on a django TestCase (for example for debugging purposes), results unintended behavior, such as certain methods not being called. Most importantly _fixture_callback and _fixture_teardown which also perform database rollback (or call flush).

References:
https://github.com/pytest-dev/pytest/pull/1890
https://github.com/pytest-dev/pytest/issues/1932
https://github.com/pytest-dev/pytest-django/pull/406
https://github.com/python/cpython/blob/master/Lib/unittest/case.py#L651-L658

Change History (13)

comment:1 Changed 2 years ago by Tim Graham

Component: UncategorizedTesting framework
Type: UncategorizedNew feature

Would this have benefit outside of the usage of third-party test runners such as pytest? If so, what does it look like?

comment:2 Changed 2 years ago by Pavel Savchenko

Well, not that I know of yet, but basically, any test runner that allows unittests may assume "debug" can be used for exception debugging; and right now this cannot be done without issues when using Django's SimpleTestCase derivatives.

Regardless, the patch should be fairly simple 10 line method and I can submit that. It's invoking the test that I'm having trouble with, how to invoke "debug" as part of the regular test suite, maybe we can use call_command...

Last edited 2 years ago by Pavel Savchenko (previous) (diff)

comment:3 Changed 2 years ago by Tim Graham

Triage Stage: UnreviewedAccepted

comment:4 Changed 2 years ago by Pavel Savchenko

As for other test runners, here's now nose2 invokes a debugger (IIRC pytest does something similar):
https://github.com/nose-devs/nose2/blob/master/nose2/plugins/debugger.py#L41-L60

This means that the post_mortem is invoked only after the exception was already caught in SimpleTestCase.__call__ and, for example, the database was already rolled back. The test frameworks simple have no real way to examine the state of the application at the time of the error (but only "after", when it may be too late).

comment:5 Changed 2 years ago by Pavel Savchenko

Has patch: set

Added test case (demonstrating usage expectations) and resolution in PR 7436

comment:6 Changed 2 years ago by Adam (Chainz) Johnson

Cc: me@… added

comment:7 Changed 20 months ago by Tim Graham

Patch needs improvement: set

I think the test could be a bit cleaner. I sent some edits to the PR, though I broke some things too.

comment:8 Changed 13 months ago by Tim Graham

Patch needs improvement: unset

comment:9 Changed 10 months ago by Tim Martin

Patch needs improvement: set

comment:10 Changed 6 months ago by Asif Saifuddin Auvi

Needs documentation: set
Version: 1.10master

comment:11 in reply to:  10 Changed 6 months ago by Pavel Savchenko

Patch needs improvement: unset

Replying to Asif Saifuddin Auvi:

Patch was improved as per Tim Martin's notes in commit 7ec5fae. I added a sketch for release note mention in this comment.

Is there other documentation you had in mind?

comment:12 Changed 6 months ago by Tim Graham

Needs documentation: unset

comment:13 Changed 6 months ago by Tim Graham

Patch needs improvement: set
Note: See TracTickets for help on using tickets.
Back to Top