Opened 5 years ago

Closed 5 years ago

Last modified 4 years ago

#12399 closed (fixed)

memcached not working as expected when setting keys with a timeout > 30 days

Reported by: houdinihound Owned by: nobody
Component: Core (Cache system) Version: master
Severity: Keywords: cached_db session memcached
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

Using the 'django.contrib.sessions.backends.cached_db' and setting SESSION_COOKIE_AGE to any number of seconds greater than 2592000 (i.e. > 30 days) with memcached does not to work as intended in that the session info is never sent back from the cache.

Memcached takes an 'expire_in_seconds' argument which can be up to 2592000 (30 days). For numbers greater than that it requires a Unix timestamp to set the expiry date.

Currently cached_db.py passes in settings.SESSION_COOKIE_AGE which is always a number of seconds. Consequently this works up to 30 days (2592000 seconds) but if SESSION_COOKIE_AGE is greater than that it does not.

To illustrate:
Memcached output with SESSION_COOKIE_AGE = 2592000 (i.e. 30 days):

<280 server listening (udp)
<288 new client connection
<288 get 4c35074ef4038b16a2db4f728324206a
>288 END
<288 set 4c35074ef4038b16a2db4f728324206a 1 2592000 334
>288 STORED
<288 connection closed.
<288 new client connection
<288 get 4c35074ef4038b16a2db4f728324206a
>288 sending key 4c35074ef4038b16a2db4f728324206a
>288 END
<288 connection closed.
<288 new client connection
<288 get 4c35074ef4038b16a2db4f728324206a
>288 sending key 4c35074ef4038b16a2db4f728324206a
>288 END

Works. After the initial store, the key is requested and sent as expected.

Now with SESSION_COOKIE_AGE = 2592001 (i.e. 30 days + 1 second):

<288 new client connection
<288 get 4c35074ef4038b16a2db4f728324206a
>288 END
<288 set 4c35074ef4038b16a2db4f728324206a 1 2592001 337
>288 STORED
<288 connection closed.
<288 new client connection
<288 get 4c35074ef4038b16a2db4f728324206a
>288 END
<288 set 4c35074ef4038b16a2db4f728324206a 1 2592001 337
>288 STORED
<288 connection closed.
<288 new client connection
<288 get 4c35074ef4038b16a2db4f728324206a
>288 END
<288 set 4c35074ef4038b16a2db4f728324206a 1 2592001 337
>288 STORED
<288 connection closed.

The key is initially stored, but each time it is requested it doesn't send the data but rather gets it from the db and keeps re-setting the same key each time.

I've patched load and save in cached_db.py to:
1) Check if memcached is being used
2) If so, check if settings.SESSION_COOKIE_AGE > 2592000
3) If so, add settings.SESSION_COOKIE_AGE to the unix timestamp (time.time()) and pass that to memcached.

which now works as expected when SESSION_COOKIE_AGE > 2592000:

<292 new client connection
<292 get 4c35074ef4038b16a2db4f728324206a
>292 END
<292 set 4c35074ef4038b16a2db4f728324206a 1 1263689307 337
>292 STORED
<292 connection closed.
<292 new client connection
<292 get 4c35074ef4038b16a2db4f728324206a
>292 sending key 4c35074ef4038b16a2db4f728324206a
>292 END
<292 connection closed.
<292 new client connection
<292 get 4c35074ef4038b16a2db4f728324206a
>292 sending key 4c35074ef4038b16a2db4f728324206a
>292 END
<292 connection closed.

Attachments (4)

patch1.diff (1.9 KB) - added by houdinihound 5 years ago.
memcached-timeout-fixes.diff (2.1 KB) - added by gciotta 5 years ago.
Fixes at cache layer, tested on 1.1.1 and trunk r12157
django_memcache.patch (1.3 KB) - added by minmax 5 years ago.
fix in cache.backend level
memcached-timeout-fixes-with-tests-r12394.diff (2.9 KB) - added by gciotta 5 years ago.
With tests, tested against today's r12394

Download all attachments as: .zip

Change History (15)

Changed 5 years ago by houdinihound

comment:1 Changed 5 years ago by houdinihound

  • Component changed from Uncategorized to Cache system
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

Changed 5 years ago by gciotta

Fixes at cache layer, tested on 1.1.1 and trunk r12157

comment:2 Changed 5 years ago by gciotta

The patch above may be better because it works at the cache layer, hence fixing django.contrib.sessions.backends.cache, django.contrib.sessions.backends.cached_db, and in general, all the cache.set() calls with timeouts greater than 30 days.

comment:3 Changed 5 years ago by gciotta

  • Summary changed from cached_db session backend and memcached not working as expected with expiry > 30 days to memcached not working as expected when setting keys with a timeout > 30 days

comment:4 Changed 5 years ago by houdinihound

gciotta's patch seems to be a much better solution to this issue. Please ignore patch1.diff - too limited in scope.

comment:5 Changed 5 years ago by gciotta

Just to make the thing clearer: The issue raises when you set a key on memcached with a timeout > 30 days. In this case, memcache will happily consider it an absolute timestamp rather than a relative offset (as the cache layer is expecting), and your keys will expire immediately.

Changed 5 years ago by minmax

fix in cache.backend level

comment:6 Changed 5 years ago by minmax

ups sorry, memcached-timeout-fixes.diff is better then django_memcache.patch

Changed 5 years ago by gciotta

With tests, tested against today's r12394

comment:7 Changed 5 years ago by gciotta

  • milestone set to 1.2
  • Version changed from 1.1 to SVN

comment:8 Changed 5 years ago by russellm

  • Triage Stage changed from Unreviewed to Ready for checkin

comment:9 Changed 5 years ago by russellm

  • Resolution set to fixed
  • Status changed from new to closed

(In [12408]) Fixed #12399 -- Added handling for memcache timeouts longer than 30 days. Thanks to houdinihound for the report, and gciotta for the patch.

comment:10 Changed 5 years ago by russellm

(In [12412]) [1.1.X] Fixed #12399 -- Added handling for memcache timeouts longer than 30 days. Thanks to houdinihound for the report, and gciotta for the patch.

Backport of r12408 from trunk.

comment:11 Changed 4 years ago by jacob

  • milestone 1.2 deleted

Milestone 1.2 deleted

Note: See TracTickets for help on using tickets.
Back to Top