Code

Opened 6 years ago

Closed 3 years ago

#9324 closed (wontfix)

cmemcache and python-memcache hashing differences

Reported by: toxik Owned by: nobody
Component: Core (Cache system) Version: master
Severity: Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

There needs to be support for decidedly using either only python-memcached or cmemcache.

The reason being that the two libraries hash different ways - if you have a box with cmemcache and two without, one of those boxes will store sessions on a different memcached than the other two.

I'm not sure what to do about it, but Malcolm wants documentation and/or a setting with which you can tell Django "use only this library" and fail if it isn't found.

ML thread: http://groups.google.com/group/django-developers/t/307825c92ca5bffa

Arguably, this should be something that the two libraries come together on, but at least solving it on our part would be nice.

It might be relevant that I've made a hackesque module which monkey patches python-memcached to hash the same way as cmemcache, see http://pypi.python.org/pypi/cmemcache_hash

Attachments (0)

Change History (9)

comment:1 Changed 6 years ago by toxik

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Summary changed from cmemcache and python-memcache to cmemcache and python-memcache hashing differences

comment:2 Changed 6 years ago by taleinat

What about having Django's memcached cache backend implement its own
hashing algorithm? This could be the "standard" one used in
libmemcache and cmemcache, or perhaps a consistent hashing algorithm
[1] such as libketama[2].

[1]http://www.socialtext.net/memcached/index.cgi?
faq#what_is_a_consistent_hashing_client

[2]http://www.last.fm/user/RJ/journal/2007/04/10/rz_libketama_-
_a_consistent_hashing_algo_for_memcache_clients

comment:3 Changed 6 years ago by taleinat

What about having Django's memcached cache backend implement its own hashing algorithm? This could be the "standard" one used in libmemcache and cmemcache, or perhaps a consistent hashing algorithm such as libketama.

comment:4 Changed 6 years ago by toxik

Another, new algorithm is exactly what we don't need. But I don't think that was what you were suggesting.

As for ketama hashing, ketama is in no way the only consistent hashing algorithm out there and while Python bindings exist, I'm more in favor of waiting for the memcached libraries themselves to get this functionality. Though some have, like pylibmc.

I think the best thing would be for Django to:

  1. Offer the alternative of forcing usage of one library.
  2. Default to trying to import python-memcached with cmemcache_hash so as to interoperate with other libraries.

And lastly, I think the mailing list is the preferred medium for discussion. :-)

comment:5 Changed 5 years ago by jacob

  • Triage Stage changed from Unreviewed to Accepted

comment:6 Changed 5 years ago by toxik

Good news, everyone! The maintainer of python-memcached said he'd include my cmemcache_hash with the distribution or something, so this will no longer be as dangerous as it was.

Question is, is it still desirable to be able to explicitly set one or the other? I mean cmemcache can kill FastCGI workers and stuff... Seen it happen. I would still appreciate the option.

comment:7 follow-up: Changed 3 years ago by anonymous

I think we can resolve this as invalid for 1.4 and wontfix for 1.3 with the deprecation of cmemcache and the requirement for explicit backend setting.

comment:8 in reply to: ↑ 7 Changed 3 years ago by otherjacob

Replying to anonymous:

I think we can resolve this as invalid for 1.4 and wontfix for 1.3 with the deprecation of cmemcache and the requirement for explicit backend setting.

Also, that would be me not realizing I wasn't logged in.

comment:9 Changed 3 years ago by otherjacob

  • Resolution set to wontfix
  • Status changed from new to closed

cmemcache is being deprecated in 1.3 and removed in 1.4.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.