#34210 closed New feature (fixed)

Show the duration of individual tests

Reported by: Paolo Melchiorre Owned by: David Smith
Component: Testing framework Version: dev
Severity: Normal Keywords: test, timing, durations
Cc: Adam Johnson, David Smith, Ruchir Harbhajanka Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I propose to add the possibility to show the duration of individual tests in the final report by ordering them from slowest to fastest and filtering only the slowest tests with a threshold.

Example:

$ python -m manage test --durations=10
Found 92 test(s).
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
Creating test database for alias 'default'...
..........
----------------------------------------------------------------------
Ran 92 tests in 1.297s

OK
Destroying test database for alias 'default'...

Slowest 10 test duration durations:
0.3597s test_detail_view_with_a_future_poll (polls.tests.PollIndexDetailTests)
0.0284s test_detail_view_with_a_past_poll (polls.tests.PollIndexDetailTests)
0.0068s test_index_view_with_a_future_poll (polls.tests.PollViewTests)
0.0047s test_index_view_with_a_past_poll (polls.tests.PollViewTests)
0.0045s test_index_view_with_two_past_polls (polls.tests.PollViewTests)
0.0041s test_index_view_with_future_poll_and_past_poll (polls.tests.PollViewTests)
0.0036s test_index_view_with_no_polls (polls.tests.PollViewTests)
0.0003s test_was_published_recently_with_future_poll (polls.tests.PollMethodTests)
0.0002s test_was_published_recently_with_recent_poll (polls.tests.PollMethodTests)
0.0002s test_was_published_recently_with_old_poll (polls.tests.PollMethodTests)

pytest has a very similar option that helps you find slower tests that you want to optimize their timing.
https://pytest.org/en/latest/how-to/usage.html#durations

There's an external package django-slow-tests that adds a similar functionality, but the code is very old and with very little test coverage, furthermore the last release was almost 4 years ago and the officially supported versions of Django are very old.
https://github.com/realpython/django-slow-tests

I propose to add this feature directly into Django tests to help users optimize their tests and also to help Django developers optimize the test of Dajngo itself.

Change History (22)

comment:1 by Carlton Gibson, 23 months ago

Cc: Adam Johnson added
Triage Stage: UnreviewedAccepted

I think this would be great. (I'm sure Adam mentioned similar once upon a time… 🤔)

I could do with the same for Django's own test suite — there's one test that I notice sitting there, which would be good to get a bead on 🙂

in reply to:  1 comment:2 by Paolo Melchiorre, 23 months ago

Replying to Carlton Gibson:

I think this would be great. (I'm sure Adam mentioned similar once upon a time… 🤔)

Sure, Adam talked about the two alternatives in his book.

Ever since I read this, I started wondering if it might not have been a good idea to have this function directly in Django.

I could do with the same for Django's own test suite — there's one test that I notice sitting there, which would be good to get a bead on 🙂

By often running Django tests locally, lately I too have noticed that for a particular test the tests remain pending for a few seconds and it would not hurt to find out which one it is and optimize it.

comment:3 by Mariusz Felisiak, 23 months ago

As far as I'm aware, we declined to add --durations in this PR.

comment:4 by Carlton Gibson, 23 months ago

I think last comment there was mine here https://github.com/django/django/pull/13224#issuecomment-664913219 — pro durations but handling it separately, so here maybe :)

comment:5 by Adam Johnson, 23 months ago

django-rich has an open PR from David Smith to add test timing functionality: https://github.com/adamchainz/django-rich/pull/30

It’s stalled because of sorting out parallel testing, and trying to also time setUp / tearDown / setUpClass / tearDownClass.

comment:6 by Paolo Melchiorre, 23 months ago

Seems to me like a very useful feature to have right in Django tests.

It would be very useful to be able to use this feature also to optimize Django's tests themselves.

It could be a feature that a tool like Django Rich could add color to?

comment:7 by Adam Johnson, 23 months ago

Yes, though the initial idea was to implement it first in a third party package before proposing it for Django itself.

comment:8 by Paolo Melchiorre, 23 months ago

I think it's a good idea.

Last edited 23 months ago by shaswat (previous) (diff)

comment:9 by shaswat, 23 months ago

Owner: changed from nobody to shaswat
Status: newassigned

comment:10 by David Smith, 22 months ago

Cc: David Smith added

comment:11 by Ruchir Harbhajanka, 19 months ago

Cc: Ruchir Harbhajanka added

comment:12 by David Smith, 19 months ago

Python 3.12 will add a --durations flag.

https://github.com/python/cpython/pull/12271

I am leaning towards saying thats enough to call this wontfix.

While it's likely that a bit more investigation is needed to see if it covers SetUp / TearDown I wonder if its reasonable to say let's implement that additional functionality in a 3rd party package first?

comment:13 by Adam Johnson, 19 months ago

That sounds good. Even with the 3.12 support, Django will still need to forward the option through to unittest.

comment:14 by Adam Johnson, 19 months ago

Oh, and the unittest flag will not work with parallel testing - there will need to be some work in Django to merge the test times across processes.

For future reference David opened a PR in django-rich for timing: https://github.com/adamchainz/django-rich/pull/30 . It stalled due to my prefectionism trying to time all setups and teardowns...

comment:15 by David Smith, 17 months ago

Owner: changed from shaswat to David Smith

comment:16 by David Smith, 17 months ago

Has patch: set

comment:17 by Sarah Boyce, 17 months ago

I think this also will close this ticket: https://code.djangoproject.com/ticket/31827

in reply to:  17 comment:18 by Mariusz Felisiak, 17 months ago

Replying to Sarah Boyce:

I think this also will close this ticket: https://code.djangoproject.com/ticket/31827

Probably yes, let's wait for the final release first.

comment:19 by Mariusz Felisiak, 17 months ago

#31827 was a duplicate for showing N slowest tests.

comment:20 by Natalia Bidart, 15 months ago

Patch needs improvement: set

Setting the flag "patch needs improvement" to remove this work from the review queue (and also because the branch will need a rebase and conflict fixes before landing).

comment:21 by Mariusz Felisiak, 14 months ago

Patch needs improvement: unset
Triage Stage: AcceptedReady for checkin

comment:22 by Mariusz Felisiak <felisiak.mariusz@…>, 14 months ago

Resolution: fixed
Status: assignedclosed

In 74b50741:

Fixed #34210 -- Added unittest's durations option to the test runner.

Note: See TracTickets for help on using tickets.
Back to Top