Opened 8 years ago

Closed 6 years ago

#9324 closed (wontfix)

cmemcache and python-memcache hashing differences

Reported by: Ludvig Ericson Owned by: nobody
Component: Core (Cache system) Version: master
Severity: Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

There needs to be support for decidedly using either only python-memcached or cmemcache.

The reason being that the two libraries hash different ways - if you have a box with cmemcache and two without, one of those boxes will store sessions on a different memcached than the other two.

I'm not sure what to do about it, but Malcolm wants documentation and/or a setting with which you can tell Django "use only this library" and fail if it isn't found.

ML thread: http://groups.google.com/group/django-developers/t/307825c92ca5bffa

Arguably, this should be something that the two libraries come together on, but at least solving it on our part would be nice.

It might be relevant that I've made a hackesque module which monkey patches python-memcached to hash the same way as cmemcache, see http://pypi.python.org/pypi/cmemcache_hash

Change History (9)

comment:1 Changed 8 years ago by Ludvig Ericson

Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset
Summary: cmemcache and python-memcachecmemcache and python-memcache hashing differences

comment:2 Changed 8 years ago by taleinat

What about having Django's memcached cache backend implement its own
hashing algorithm? This could be the "standard" one used in
libmemcache and cmemcache, or perhaps a consistent hashing algorithm
[1] such as libketama[2].

[1]http://www.socialtext.net/memcached/index.cgi?
faq#what_is_a_consistent_hashing_client

[2]http://www.last.fm/user/RJ/journal/2007/04/10/rz_libketama_-
_a_consistent_hashing_algo_for_memcache_clients

comment:3 Changed 8 years ago by taleinat

What about having Django's memcached cache backend implement its own hashing algorithm? This could be the "standard" one used in libmemcache and cmemcache, or perhaps a consistent hashing algorithm such as libketama.

comment:4 Changed 8 years ago by Ludvig Ericson

Another, new algorithm is exactly what we don't need. But I don't think that was what you were suggesting.

As for ketama hashing, ketama is in no way the only consistent hashing algorithm out there and while Python bindings exist, I'm more in favor of waiting for the memcached libraries themselves to get this functionality. Though some have, like pylibmc.

I think the best thing would be for Django to:

  1. Offer the alternative of forcing usage of one library.
  2. Default to trying to import python-memcached with cmemcache_hash so as to interoperate with other libraries.

And lastly, I think the mailing list is the preferred medium for discussion. :-)

comment:5 Changed 8 years ago by Jacob

Triage Stage: UnreviewedAccepted

comment:6 Changed 7 years ago by Ludvig Ericson

Good news, everyone! The maintainer of python-memcached said he'd include my cmemcache_hash with the distribution or something, so this will no longer be as dangerous as it was.

Question is, is it still desirable to be able to explicitly set one or the other? I mean cmemcache can kill FastCGI workers and stuff... Seen it happen. I would still appreciate the option.

comment:7 Changed 6 years ago by anonymous

I think we can resolve this as invalid for 1.4 and wontfix for 1.3 with the deprecation of cmemcache and the requirement for explicit backend setting.

comment:8 in reply to:  7 Changed 6 years ago by otherjacob

Replying to anonymous:

I think we can resolve this as invalid for 1.4 and wontfix for 1.3 with the deprecation of cmemcache and the requirement for explicit backend setting.

Also, that would be me not realizing I wasn't logged in.

comment:9 Changed 6 years ago by otherjacob

Resolution: wontfix
Status: newclosed

cmemcache is being deprecated in 1.3 and removed in 1.4.

Note: See TracTickets for help on using tickets.
Back to Top