Opened 12 years ago

Closed 12 years ago

Last modified 11 years ago

#15324 closed (fixed)

memcached cache backend creating new connection for every action

Reported by: J. David Lowe Owned by: nobody
Component: Core (Cache system) Version: 1.3-beta
Severity: Keywords: blocker regression
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

r15005 introduced a problem for MemcachedCache caches: every call to any cache method which accesses the cache (add, get, set, delete, get_many, incr, decr, set_many, delete_many, clear and even close!) opens a brand-new connection to memcached via the self._lib.Client(...) call now hidden in self._cache. I think this is a major problem for busy sites who are aggressively caching: our site was brought down when we upgraded to 1.3_beta_1, as it very quickly exceeded the memcached concurrency limit.

I've attached a patch including a minimal test case.

I ran the tests against django trunk r15560, with python_memcached-1.44-py2.6.egg and python 2.6.5.

Attachments (1)

memcached_connection_reuse.patch (2.2 KB) - added by J. David Lowe 12 years ago.

Download all attachments as: .zip

Change History (5)

Changed 12 years ago by J. David Lowe

comment:1 Changed 12 years ago by Russell Keith-Magee

Keywords: blocker regression added
Triage Stage: UnreviewedReady for checkin

comment:2 Changed 12 years ago by Russell Keith-Magee

There's a slight problem with the patch as presented -- I can't get the test case to fail (using python-memcached-1.45). I suspect this may have something to do with the internals of python-memcache, and its own internal connection reuse policies.

However, there is a clear behavioral difference between Django 1.2 and trunk with regards to connection creation, and fixing the problem is fairly trivial, so I'm willing to take this as an untested feature. If anyone manages to find a way to test this in a consistent way, I'm happy to apply the patch post-facto.

comment:3 Changed 12 years ago by Russell Keith-Magee

Resolution: fixed
Status: newclosed

In [15567]:

Fixed #15324 -- Modified the connection creation code for the memcache cache backend. Thanks to dlowe for the report and initial patch.

comment:4 Changed 11 years ago by Jacob

milestone: 1.3

Milestone 1.3 deleted

Note: See TracTickets for help on using tickets.
Back to Top