Opened 15 years ago

Closed 11 years ago

#11011 closed New feature (duplicate)

Memcached backend: pass parameters to memcache backend

Reported by: erny Owned by: otherjacob
Component: Core (Cache system) Version: 1.0
Severity: Normal Keywords: memcached
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

The problem:

  1. Django instantiates memcache client w/o parameters, i.e. pickleProtocol=0
  2. I compress a string to < 512K, binary pickle format
  3. I encapsulate it inside a dummy object to prevent getting it converted to unicode
  4. I pass it to cache.set
  5. django does the set w/o min_compress_len.
  6. Then python-memcache.set does (indirectly) o dumps with pickleProtocol=0, which results in a my < 512K to a >1,4M string and is silently discarted as max is 1 MB.

Possible solutions:

  • django memcached passes pickleProtocol option to memcached backend client
  • django memcached does implement splitting into chunks
  • I split it into smaller parts in the dummy objects

Change History (8)

comment:1 by Alex Gaynor, 15 years ago

Triage Stage: UnreviewedAccepted

comment:2 by Malcolm Tredinnick, 14 years ago

A CACHE_OPTIONS setting, similar to DATABASE_OPTIONS, might be appropriate here. I can imagine that different settings backends requiring a number of different settings (long-term) and trying to jam them all in a URI gets ugly fast. I wouldn't stand in the way of a new setting here (the options would be CACHE_BACKEND specific, but that's fine when you're fine-tuning backend params).

comment:3 by otherjacob, 13 years ago

milestone: 1.4
Owner: changed from nobody to otherjacob

This should be doable, but probably too late to get into 1.3. I'll see if I can come up with a good fix for 1.4

comment:4 by Julien Phalip, 13 years ago

Severity: Normal
Type: New feature

comment:5 by Jacob, 12 years ago

milestone: 1.4

Milestone 1.4 deleted

comment:2 by Aymeric Augustin, 12 years ago

UI/UX: unset

Change UI/UX from NULL to False.

comment:3 by Aymeric Augustin, 12 years ago

Easy pickings: unset

Change Easy pickings from NULL to False.

comment:4 by Aymeric Augustin, 11 years ago

Resolution: duplicate
Status: newclosed
  • #19810 proposes using the highest pickle protocol available, which is going to resolve the root cause — pickling with an inefficient protocol.
  • The CACHES setting was already split like DATABASES, as suggested by Malcolm in comment 2.

I think there's nothing left in this ticket, closing as a duplicate of #19810 which is more specific and will resolve the problem.

Note: See TracTickets for help on using tickets.
Back to Top