id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
23746	Allow assertNumQueries to clear caches before it runs	Wojtek Ruszczewski	nobody	"The `assertNumQueries` manager / method may return different results depending on what is in various caches. Consequently, some tests using it rely on test ordering, particularly those that count some content type queries. For instance if you [https://code.djangoproject.com/ticket/23742 run the suite in reverse] you'll see `test_group_permission_performance` failing due to `ContentType` being or not being asked about `auth.Group` in some preceding test ^[#f1 1]^.

[https://github.com/wrwrwr/django/compare/fix/assert-num-queries-cache-dependence The proposed solution] resets ""query"" caches on entering the manager (with a switch to disable it for actual caching tests). After introducing the change 10 tests started to fail (`ContentType` cache tests and some `prefetch_related` tests), 2 specific workarounds were no longer necessary (#17377, #20432) and a failure from [https://github.com/django/django/pull/3426 PR 3426] disappeared ^[#f2 2]^.

\\

Note to prospective reviewers: Are there any other caches that should be cleared besides the `ContentType` and `Site`? Related objects caches maybe? Do the affected `prefetch_related` operations actually require content type queries? Would it be better to clear all the caches after every test case to avoid other kinds of dependencies?

![1][[=#f1]] What happens in the likewise failing with ""--reverse"" `test_user_permission_performance` is a bit intriguing and probably more involved -- two identical content type queries, savepoint, content type insertion and a release?

![2][[=#f2]] Only tested with Python 3.3 / Postgres,"	New feature	new	Testing framework	dev	Normal			Tim Graham Ülgen Sarıkavak Florian Demmer	Accepted	1	0	0	1	0	0