id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
11331	Memcached backend closes connection after every request	booink@…	Ed Morley	"Fix for http://code.djangoproject.com/ticket/5133 kills servers in production.

--

Copy of http://code.djangoproject.com/ticket/5133#comment:16

The patch takes care of connections kept open, but it introduces another problem - the need to open one or more tcp connections every single request.

With a simple loop, you can make a system run out of sockets easily - after a socket is closed, that port cannot be reused for an eternity, ranging from 1 minute to 4 depending on OS. If enough sockets get stuck in TIME_WAIT state, the server simply fails to connect to memcached and start serving everything from db again - that's not something you want to see on a site with sufficient traffic to need a memcached installation.

In my opinion, the cure is worse than the disease. There's an easy workaround available for the original problem: restart workers after a certain amount of requests. With max-request=500 on a 5 threads deamon process (mod_wsgi, times 20 processes), we never go over 100 connections on our memcached server, started with the default cap of 1024 connections. If you run mod_python, use MaxRequestsPerChild?.

My current solution is to just noop the whole fix with one line in any .py: django.core.cache.backends.memcached.CacheClass?.close = lambda x: None. It might be an idea to make it configurable so people can choose between disconnect after every request and keep it open until process restart.
"	Cleanup/optimization	closed	Core (Cache system)	1.0	Normal	fixed	pylibmc	mjmalone@… daevaorn@… harm.verhagen+django@… Rachel Willmer trbs@… jeremy.orem@… emorley@…	Accepted	1	0	0	0	0	0