Opened 17 years ago

Closed 16 years ago

#3971 closed (wontfix)

Verbose testing framework does not say which test it is about to execute

Reported by: Malcolm Tredinnick Owned by: nobody
Component: Testing framework Version: dev
Severity: Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Some of the changes I'm making at the moment have the side-effect of breaking tests very early in their execution (when table flushing fails, for example). The problem is, even when running with -v2, I get no indication of which test it was trying to run. The verbose output only seems shows output that didn't cause the harness itself to fail.

I'm not sure how feasible it is to implement, but it would be nice for times to like this to have a -v3 option that said something like "About to attempt modeltests.model_forms.models", so that if I break the harness itself, I know where to look. I had to resort to running each directory in tests/modeltests/ one at a time until I saw the breakage (tests aren't run in any particularly predictable order, that I can see). Not the end of the world, but time consuming and a little tiring the third or fourth time it happened.

Attachments (1)

3971.diff (2.6 KB ) - added by Malcolm Tredinnick 17 years ago.
Updated patch to current svn

Download all attachments as: .zip

Change History (5)

comment:1 by Simon G. <dev@…>, 17 years ago

Triage Stage: UnreviewedAccepted

comment:2 by Russell Keith-Magee, 17 years ago

A vanilla unit test under -v2 *does* report the name of the test before the test is executed. The problem you are seeing is caused by the way we have overloaded TestCase.run() to call the flush and fixture load code before calling the baseclass run() implementation. As a result, the flush is called before the 'start test' debug message is printed.

I've attached a patch which I think will fix the problem. Could you try it out with one of your failing tests and let me know if it works?

Interesting side effect - this patch can be cleaned up to remove the need for django.test.TestCase. This approach would allow any TestCase instance with a 'fixtures' member variable to operate as a 'flush and load' test case. I'm not sure how I feel about this; do you have any opinions?

by Malcolm Tredinnick, 17 years ago

Attachment: 3971.diff added

Updated patch to current svn

comment:3 by Malcolm Tredinnick, 17 years ago

Finally remembered to test this. It doesn't work as I would have thought, but it's possible my particular stress testing isn't worth solving.

I simulated my previous problems by putting "raise Exception" into core/management.py on line 732 (just before the call to load_data() in the flush() function). Then, running all the tests under -v2 exhibits the expected explosion, but we aren't told what failed.

Thinking about this a bit more, I'm not sure it's worth going to extreme lengths to account for this behaviour. Detecting that some bozo has stuck a monkey wrench deep in the guts of management.py might not be easy.

In any case, I've updated the patch to apply against recent trunk, since there were some changes in the interim and a bit of manual patching was required.

comment:4 by Russell Keith-Magee, 16 years ago

Resolution: wontfix
Status: newclosed

Closing this off; Malcolm has questioned the need for a fix, and we're no longer overloading TestCase.run() anyway.

Note: See TracTickets for help on using tickets.
Back to Top