Opened 15 years ago

Last modified 7 years ago

#11505 new New feature

Django's TestCase should reset the cache

Reported by: andrewfong Owned by: nobody
Component: Testing framework Version: dev
Severity: Normal Keywords: cache testing flush
Cc: Mikhail Korobov, jim.dalton@…, hv@…, someuniquename@…, hirokiky@… Triage Stage: Accepted
Has patch: yes Needs documentation: yes
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

In between test cases, the cache should be flushed in order to prevent one test case from polluting another, similar to how the database is currently flushed (or transactions rolled back) in Django's test case.

This will be easier to implement if #11503 is complete

Attachments (4)

patch.diff (1.9 KB ) - added by andrewfong 15 years ago.
Adds cache flushing after every testcase + a simple regression test
doc_patch.patch (722 bytes ) - added by andrewfong 14 years ago.
Documentation of cache flushing
restore_cache_during_tests.diff (14.8 KB ) - added by Jim Dalton 13 years ago.
restore_cache_during_tests.v2.diff (20.6 KB ) - added by Jim Dalton 12 years ago.

Download all attachments as: .zip

Change History (27)

comment:2 by Russell Keith-Magee, 15 years ago

Triage Stage: UnreviewedAccepted

by andrewfong, 15 years ago

Attachment: patch.diff added

Adds cache flushing after every testcase + a simple regression test

comment:3 by andrewfong, 15 years ago

Has patch: set
Needs documentation: set

by andrewfong, 14 years ago

Attachment: doc_patch.patch added

Documentation of cache flushing

comment:4 by andrewfong, 14 years ago

Needs documentation: unset

comment:5 by Mikhail Korobov, 14 years ago

Cc: Mikhail Korobov added

There is a pattern to run tests on production machine. With this patch e.g. production memcached instance can be flushed after every commit. Databases are different because one DB server can handle several databases and test database can be created during test setup. I think it can't be done for 'test memcached instance'.

Memcached key space can be easily shared between production and test environment using custom cache backend that prepends all key with fixed string (giving us an equivalent of 'test database' inside single memcached instance). But invalidation is nearly impossible.

comment:6 by Julien Phalip, 13 years ago

Severity: Normal
Type: Bug

comment:7 by Julien Phalip, 13 years ago

Easy pickings: unset
UI/UX: unset

#16401 is a duplicate and contains a much more thorough patch.

comment:8 by Julien Phalip, 13 years ago

Type: BugNew feature

comment:9 by Julien Phalip, 13 years ago

Needs documentation: set
Needs tests: set
Patch needs improvement: set

in reply to:  7 comment:10 by Jim Dalton, 13 years ago

Cc: jim.dalton@… added
Needs tests: unset

Replying to julien:

#16401 is a duplicate and contains a much more thorough patch.

Thanks julien for consolidating the new ticket I opened back to this one.

I'm uploading the latest patch from that ticket here. In a nutshell, this approach monkey patches cache and get_cache during test setup and keeps track of any keys used during the test. After each test, those keys are flushed. We also prepend '_test' to the key prefix to ensure existing values are not overwritten. The patch is tested and working. I'm leaving it as "needs improvement" for now mainly because there is one test in the suite that is failing and I haven't determined why yet. It's minor whatever is is though.

In sum, the patch works great and solves the problem. There was a bit of discussion here about the approach, however: https://groups.google.com/forum/#!topic/django-developers/zlaPsP13dUY . Jacob stated he wasn't necessarily a fan of monkey patching and suggested an alternative. I had some thoughts which I posted in response as well.

If anyone reading this comment is interested in help push this ticket toward resolution, a great way would be to review this patch and then weigh in on that django dev discussion. If we can reach a consensus on django-dev I'd be happy to put in some time to implement whatever approach is agreed upon.

by Jim Dalton, 13 years ago

comment:11 by Jim Dalton, 12 years ago

Patch needs improvement: unset

Adding a new patch for this, which extends the modified cache approach to cache middleware. I had noticed on some of my projects that certain tests were failing when using cache middleware, which turned out to be because the cache was returning the response and thus the context was not included in the response to the test client. Tests reproducing this error state as well as a fix are included in the latest patch. The full test suite passes as well (minus 5 unrelated failures that are on trunk right now).

The failures to me are a good illustration of why this patch is important. The system should ideally be as clean as possible in between each test; cache values carrying over from test to test create problematic behavior when a full test suite is run.

I might post again to the dev list if there's no movement on this. If anyone on the CC list here would like to review this patch and give any feedback please do so. At this point, all that's really required from what I can see is a few notes in the documentation.

by Jim Dalton, 12 years ago

comment:12 by Aymeric Augustin, 12 years ago

In [17042]:

Fixed several problems that hid one another in the cache tests and code.

1 - Used django.test.TestCase instead of unittest.TestCase so that the override_settings decorator works.
2 - Reverted parts of r17039 that caused failures (masked until 1).
3 - Isolated tests by clearing the cache in tear down (masked until 1). Refs #11505.
4 - Fixed a bug in cache key generation (revealed by 3).
5 - Fixed a test that relied on this bug -- hardcoding the generated cache keys in tests isn't a very good idea anyway.
6 - Uniformized some parts of the cache tests.

comment:13 by Thomas Güttler, 12 years ago

Cc: hv@… added

comment:14 by Evstifeev Roman, 11 years ago

Cc: someuniquename@… added

comment:15 by Hiroki Kiyohara, 10 years ago

Cc: hirokiky@… added

comment:16 by Thomas C, 9 years ago

I ran into another example of tests depending on each other execution, because of the way django.contrib.sites.models.Site instances are cached in a global variable SITE_CACHE

Reproductible with the command python tests/runtests.py contenttypes_tests.tests.ContentTypesViewsTests.test_shortcut_with_absolute_url syndication_tests

Last edited 9 years ago by Thomas C (previous) (diff)

comment:17 by Tim Graham <timograham@…>, 9 years ago

In 3bb78c5e7a330ed70c518e77899bf8a5ac6bf766:

Cleanup cache in contrib.sites to prevent test interference -- refs #11505

comment:18 by Tim Graham <timograham@…>, 9 years ago

In 4135d837027eac43ec416856d9476c478167d8a6:

Isolated a flatpages test; refs #11505.

comment:19 by Tim Graham <timograham@…>, 9 years ago

In 1806e059f6127e3e4572db09f297984c96ea9d02:

[1.8.x] Isolated a flatpages test; refs #11505.

Backport of 4135d837027eac43ec416856d9476c478167d8a6 from master

comment:20 by Tim Graham <timograham@…>, 9 years ago

In e0b3926026984dccc86a09c0c4f32e8bec6f3fe1:

Isolated auth_tests from contenttypes_tests; refs #11505.

comment:21 by Tim Graham <timograham@…>, 9 years ago

In 259259a819a12a9ecfd0448c646899fad78e23e0:

[1.8.x] Isolated auth_tests from contenttypes_tests; refs #11505.

Backport of e0b3926026984dccc86a09c0c4f32e8bec6f3fe1 from master

comment:22 by Tim Graham <timograham@…>, 9 years ago

In f668bac9d223408627ca92b2281cf7110039510b:

Fixed #24345 -- Isolated sitemaps_tests from contenttypes_tests; refs #11505

comment:23 by Tim Graham <timograham@…>, 9 years ago

In bc2eb6bfef00fd8ea4395e3462366b6571f88601:

[1.8.x] Fixed #24345 -- Isolated sitemaps_tests from contenttypes_tests; refs #11505

Backport of f668bac9d223408627ca92b2281cf7110039510b from master

comment:24 by Tim Graham <timograham@…>, 8 years ago

In 2a9bcb5:

Refs #26085, #11505 -- Cleared Site cache in contenttypes_tests.

Note: See TracTickets for help on using tickets.
Back to Top