Opened 7 years ago

Closed 7 years ago

Last modified 5 years ago

#12399 closed (fixed)

memcached not working as expected when setting keys with a timeout > 30 days

Reported by: houdinihound Owned by: nobody
Component: Core (Cache system) Version: master
Severity: Keywords: cached_db session memcached
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

Using the 'django.contrib.sessions.backends.cached_db' and setting SESSION_COOKIE_AGE to any number of seconds greater than 2592000 (i.e. > 30 days) with memcached does not to work as intended in that the session info is never sent back from the cache.

Memcached takes an 'expire_in_seconds' argument which can be up to 2592000 (30 days). For numbers greater than that it requires a Unix timestamp to set the expiry date.

Currently cached_db.py passes in settings.SESSION_COOKIE_AGE which is always a number of seconds. Consequently this works up to 30 days (2592000 seconds) but if SESSION_COOKIE_AGE is greater than that it does not.

To illustrate:
Memcached output with SESSION_COOKIE_AGE = 2592000 (i.e. 30 days):

<280 server listening (udp)
<288 new client connection
<288 get 4c35074ef4038b16a2db4f728324206a
>288 END
<288 set 4c35074ef4038b16a2db4f728324206a 1 2592000 334
>288 STORED
<288 connection closed.
<288 new client connection
<288 get 4c35074ef4038b16a2db4f728324206a
>288 sending key 4c35074ef4038b16a2db4f728324206a
>288 END
<288 connection closed.
<288 new client connection
<288 get 4c35074ef4038b16a2db4f728324206a
>288 sending key 4c35074ef4038b16a2db4f728324206a
>288 END

Works. After the initial store, the key is requested and sent as expected.

Now with SESSION_COOKIE_AGE = 2592001 (i.e. 30 days + 1 second):

<288 new client connection
<288 get 4c35074ef4038b16a2db4f728324206a
>288 END
<288 set 4c35074ef4038b16a2db4f728324206a 1 2592001 337
>288 STORED
<288 connection closed.
<288 new client connection
<288 get 4c35074ef4038b16a2db4f728324206a
>288 END
<288 set 4c35074ef4038b16a2db4f728324206a 1 2592001 337
>288 STORED
<288 connection closed.
<288 new client connection
<288 get 4c35074ef4038b16a2db4f728324206a
>288 END
<288 set 4c35074ef4038b16a2db4f728324206a 1 2592001 337
>288 STORED
<288 connection closed.

The key is initially stored, but each time it is requested it doesn't send the data but rather gets it from the db and keeps re-setting the same key each time.

I've patched load and save in cached_db.py to:
1) Check if memcached is being used
2) If so, check if settings.SESSION_COOKIE_AGE > 2592000
3) If so, add settings.SESSION_COOKIE_AGE to the unix timestamp (time.time()) and pass that to memcached.

which now works as expected when SESSION_COOKIE_AGE > 2592000:

<292 new client connection
<292 get 4c35074ef4038b16a2db4f728324206a
>292 END
<292 set 4c35074ef4038b16a2db4f728324206a 1 1263689307 337
>292 STORED
<292 connection closed.
<292 new client connection
<292 get 4c35074ef4038b16a2db4f728324206a
>292 sending key 4c35074ef4038b16a2db4f728324206a
>292 END
<292 connection closed.
<292 new client connection
<292 get 4c35074ef4038b16a2db4f728324206a
>292 sending key 4c35074ef4038b16a2db4f728324206a
>292 END
<292 connection closed.

Attachments (4)

patch1.diff (1.9 KB) - added by houdinihound 7 years ago.
memcached-timeout-fixes.diff (2.1 KB) - added by gciotta 7 years ago.
Fixes at cache layer, tested on 1.1.1 and trunk r12157
django_memcache.patch (1.3 KB) - added by Karataev Pavel 7 years ago.
fix in cache.backend level
memcached-timeout-fixes-with-tests-r12394.diff (2.9 KB) - added by gciotta 7 years ago.
With tests, tested against today's r12394

Download all attachments as: .zip

Change History (15)

Changed 7 years ago by houdinihound

Attachment: patch1.diff added

comment:1 Changed 7 years ago by houdinihound

Component: UncategorizedCache system

Changed 7 years ago by gciotta

Fixes at cache layer, tested on 1.1.1 and trunk r12157

comment:2 Changed 7 years ago by gciotta

The patch above may be better because it works at the cache layer, hence fixing django.contrib.sessions.backends.cache, django.contrib.sessions.backends.cached_db, and in general, all the cache.set() calls with timeouts greater than 30 days.

comment:3 Changed 7 years ago by gciotta

Summary: cached_db session backend and memcached not working as expected with expiry > 30 daysmemcached not working as expected when setting keys with a timeout > 30 days

comment:4 Changed 7 years ago by houdinihound

gciotta's patch seems to be a much better solution to this issue. Please ignore patch1.diff - too limited in scope.

comment:5 Changed 7 years ago by gciotta

Just to make the thing clearer: The issue raises when you set a key on memcached with a timeout > 30 days. In this case, memcache will happily consider it an absolute timestamp rather than a relative offset (as the cache layer is expecting), and your keys will expire immediately.

Changed 7 years ago by Karataev Pavel

Attachment: django_memcache.patch added

fix in cache.backend level

comment:6 Changed 7 years ago by Karataev Pavel

ups sorry, memcached-timeout-fixes.diff is better then django_memcache.patch

Changed 7 years ago by gciotta

With tests, tested against today's r12394

comment:7 Changed 7 years ago by gciotta

milestone: 1.2
Version: 1.1SVN

comment:8 Changed 7 years ago by Russell Keith-Magee

Triage Stage: UnreviewedReady for checkin

comment:9 Changed 7 years ago by Russell Keith-Magee

Resolution: fixed
Status: newclosed

(In [12408]) Fixed #12399 -- Added handling for memcache timeouts longer than 30 days. Thanks to houdinihound for the report, and gciotta for the patch.

comment:10 Changed 7 years ago by Russell Keith-Magee

(In [12412]) [1.1.X] Fixed #12399 -- Added handling for memcache timeouts longer than 30 days. Thanks to houdinihound for the report, and gciotta for the patch.

Backport of r12408 from trunk.

comment:11 Changed 5 years ago by Jacob

milestone: 1.2

Milestone 1.2 deleted

Note: See TracTickets for help on using tickets.
Back to Top