Opened 3 years ago

Closed 2 years ago

#29106 closed Cleanup/optimization (fixed)

Make manage.py test display the test tags that are in use

Reported by: Rich Jones Owned by: Sanyam Khurana
Component: Testing framework Version: 2.0
Severity: Normal Keywords:
Cc: Sanyam Khurana Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description

_This ticket is part of the Django Test Output Beautification initiative: https://code.djangoproject.com/ticket/29101_

Django 1.10 introduced a wonderful new feature, test tagging, which allows the user running tests to control which tests are included and excluded in a test execution. This is particularly handy in complicated and conditional continuous integration setups, which may require lots of different tag inclusions and exclusions.

The annoying part is that the default test output, and even the test output with higher verbosity levels, never displays which tests are excluded or included.

I'd be great if this information could be displayed in three ways during a test execution: Once during initialization, to show which tags were included or excluded, once for each test as it is skipped, and once during the final tally.

So, at the beginning:

System check identified no issues (0 silenced).
Running tests, excluding [slow, veryslow].

For each test skipped:

========================
Testing 'test_good_file_grouping'
(data_refinery_workers.downloaders.test_transcriptome_index.DownloadTranscriptomeIndexTestCase)
_Makes sure that proper file group runs the pipeline._
-----------------------------------------------------
...
Ok!

========================
Skipping 'test_good_file_grouping'
(data_refinery_workers.downloaders.test_transcriptome_index.DownloadTranscriptomeIndexTestCase)
[veryslow]
-----------------------------------------------------

And finally, at the end:

----------------------------------------------------------------------
Ran 38 tests in 34.808s. Skipped 4 tests.

This should work appropriately for all of the different tag-related test arguments.

It'd also be nice if this test output was appropriately colorized.

Related tickets:
https://code.djangoproject.com/ticket/29105
https://code.djangoproject.com/ticket/22449
https://code.djangoproject.com/ticket/29105

Change History (9)

comment:1 Changed 3 years ago by Tim Graham

Summary: Display Skipped Tests in Test Result OutputMake manage.py test display the test tags that are in use
Triage Stage: UnreviewedAccepted
UI/UX: unset

The tests are filtered by their tag before the test suite starts running, therefore displaying the skipped tests isn't feasible. Even if it were feasible, I wouldn't want that in my own use of test tagging. When running Django's test suite with --selenium, for example, there are about 40 tests that run out of hundreds in the complete test suite. Showing all the skipped tests would just clutter the output for this use case.

Making DiscoverRunner displaying the tags seems fine (at verbosity 2 or greater).

comment:2 Changed 3 years ago by Sanyam Khurana

Cc: Sanyam Khurana added
Owner: changed from nobody to Sanyam Khurana
Status: newassigned

Hey, I'm starting to work on this issue. Will submit a PR soon.

comment:3 Changed 3 years ago by Sanyam Khurana

Has patch: set

I've added a patch. Wasn't able to write test fully. Need a pointer on how to proceed. I've mentioned the issue in the PR: https://github.com/django/django/pull/9664

Last edited 3 years ago by Sanyam Khurana (previous) (diff)

comment:4 Changed 3 years ago by Tim Graham

Needs tests: set

I think you'll use django.test.utils.captured_stdout for the test.

comment:5 Changed 3 years ago by Sanyam Khurana

Needs tests: unset

Oh, yes. I don't know why I was confused about that.

That did the trick. Thank you so much. I've updated the Pull Request.

comment:6 Changed 3 years ago by Carlton Gibson

Triage Stage: AcceptedReady for checkin

comment:7 Changed 3 years ago by Tim Graham

Patch needs improvement: set
Triage Stage: Ready for checkinAccepted

comment:8 Changed 3 years ago by Sanyam Khurana

Patch needs improvement: unset

comment:9 Changed 2 years ago by Tim Graham <timograham@…>

Resolution: fixed
Status: assignedclosed

In 5a669ae2:

Fixed #29106 -- Made DiscoverRunner display selected test tags.

Note: See TracTickets for help on using tickets.
Back to Top