Opened 2 years ago
Closed 15 months ago
#34210 closed New feature (fixed)
Show the duration of individual tests
Reported by: | Paolo Melchiorre | Owned by: | David Smith |
---|---|---|---|
Component: | Testing framework | Version: | dev |
Severity: | Normal | Keywords: | test, timing, durations |
Cc: | Adam Johnson, David Smith, Ruchir Harbhajanka | Triage Stage: | Ready for checkin |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
I propose to add the possibility to show the duration of individual tests in the final report by ordering them from slowest to fastest and filtering only the slowest tests with a threshold.
Example:
$ python -m manage test --durations=10 Found 92 test(s). Creating test database for alias 'default'... System check identified no issues (0 silenced). Creating test database for alias 'default'... .......... ---------------------------------------------------------------------- Ran 92 tests in 1.297s OK Destroying test database for alias 'default'... Slowest 10 test duration durations: 0.3597s test_detail_view_with_a_future_poll (polls.tests.PollIndexDetailTests) 0.0284s test_detail_view_with_a_past_poll (polls.tests.PollIndexDetailTests) 0.0068s test_index_view_with_a_future_poll (polls.tests.PollViewTests) 0.0047s test_index_view_with_a_past_poll (polls.tests.PollViewTests) 0.0045s test_index_view_with_two_past_polls (polls.tests.PollViewTests) 0.0041s test_index_view_with_future_poll_and_past_poll (polls.tests.PollViewTests) 0.0036s test_index_view_with_no_polls (polls.tests.PollViewTests) 0.0003s test_was_published_recently_with_future_poll (polls.tests.PollMethodTests) 0.0002s test_was_published_recently_with_recent_poll (polls.tests.PollMethodTests) 0.0002s test_was_published_recently_with_old_poll (polls.tests.PollMethodTests)
pytest
has a very similar option that helps you find slower tests that you want to optimize their timing.
https://pytest.org/en/latest/how-to/usage.html#durations
There's an external package django-slow-tests
that adds a similar functionality, but the code is very old and with very little test coverage, furthermore the last release was almost 4 years ago and the officially supported versions of Django are very old.
https://github.com/realpython/django-slow-tests
I propose to add this feature directly into Django tests to help users optimize their tests and also to help Django developers optimize the test of Dajngo itself.
Change History (22)
follow-up: 2 comment:1 by , 2 years ago
Cc: | added |
---|---|
Triage Stage: | Unreviewed → Accepted |
comment:2 by , 2 years ago
Replying to Carlton Gibson:
I think this would be great. (I'm sure Adam mentioned similar once upon a time… 🤔)
Sure, Adam talked about the two alternatives in his book.
Ever since I read this, I started wondering if it might not have been a good idea to have this function directly in Django.
I could do with the same for Django's own test suite — there's one test that I notice sitting there, which would be good to get a bead on 🙂
By often running Django tests locally, lately I too have noticed that for a particular test the tests remain pending for a few seconds and it would not hurt to find out which one it is and optimize it.
comment:4 by , 2 years ago
I think last comment there was mine here https://github.com/django/django/pull/13224#issuecomment-664913219 — pro durations but handling it separately, so here maybe :)
comment:5 by , 2 years ago
django-rich has an open PR from David Smith to add test timing functionality: https://github.com/adamchainz/django-rich/pull/30
It’s stalled because of sorting out parallel testing, and trying to also time setUp / tearDown / setUpClass / tearDownClass.
comment:6 by , 2 years ago
Seems to me like a very useful feature to have right in Django tests.
It would be very useful to be able to use this feature also to optimize Django's tests themselves.
It could be a feature that a tool like Django Rich could add color to?
comment:7 by , 2 years ago
Yes, though the initial idea was to implement it first in a third party package before proposing it for Django itself.
comment:9 by , 2 years ago
Owner: | changed from | to
---|---|
Status: | new → assigned |
comment:10 by , 23 months ago
Cc: | added |
---|
comment:11 by , 20 months ago
Cc: | added |
---|
comment:12 by , 20 months ago
Python 3.12 will add a --durations flag.
https://github.com/python/cpython/pull/12271
I am leaning towards saying thats enough to call this wontfix.
While it's likely that a bit more investigation is needed to see if it covers SetUp / TearDown I wonder if its reasonable to say let's implement that additional functionality in a 3rd party package first?
comment:13 by , 20 months ago
That sounds good. Even with the 3.12 support, Django will still need to forward the option through to unittest.
comment:14 by , 20 months ago
Oh, and the unittest flag will not work with parallel testing - there will need to be some work in Django to merge the test times across processes.
For future reference David opened a PR in django-rich for timing: https://github.com/adamchainz/django-rich/pull/30 . It stalled due to my prefectionism trying to time all setups and teardowns...
comment:15 by , 18 months ago
Owner: | changed from | to
---|
comment:16 by , 17 months ago
Has patch: | set |
---|
follow-up: 18 comment:17 by , 17 months ago
I think this also will close this ticket: https://code.djangoproject.com/ticket/31827
comment:18 by , 17 months ago
Replying to Sarah Boyce:
I think this also will close this ticket: https://code.djangoproject.com/ticket/31827
Probably yes, let's wait for the final release first.
comment:20 by , 16 months ago
Patch needs improvement: | set |
---|
Setting the flag "patch needs improvement" to remove this work from the review queue (and also because the branch will need a rebase and conflict fixes before landing).
comment:21 by , 15 months ago
Patch needs improvement: | unset |
---|---|
Triage Stage: | Accepted → Ready for checkin |
I think this would be great. (I'm sure Adam mentioned similar once upon a time… 🤔)
I could do with the same for Django's own test suite — there's one test that I notice sitting there, which would be good to get a bead on 🙂