_session_cache of SessionBase affected by database session engine save

[anonymous]

I am running the exact same code on my local development machine and a production server.

Everything important I can think of is the same: both Ubuntu 13.10, both Python 2.7.5+, both Django 1.6.2, etc.

Yet the two machines are producing different behaviour!

I modified the file django/contrib/sessions/middleware.py, like so:

import sys
sys.stderr.write('\nBefore!\n')
for key, value in request.session.iteritems():
   sys.stderr.write('%s=%s\n'%(key, value))
request.session.save()
sys.stderr.write('\nAfter!\n')
for key, value in request.session.iteritems():
   sys.stderr.write('%s=%s\n'%(key, value))

So I am "printing" the session data before and after it is saved in the middleware.

On my machine, the data in question is present before and after (e.g. key = 'hello', value = '123'), but on the production server, the data in question is neither present before or after the save.

I have no idea how this is possible, but it's happening and driving me crazy.

Note: I am setting the session entry like so (in my view):

request.session['hello'] = '123'

So somewhere between that dict insertion and the middleware processing, the entry is completely disappearing, yet both machines are running identical code!

[anonymous]

Tried bringing the debug print statements forward, i.e. from the middleware to my view code (right after I insert the session entry):

import random

request.session['hello'] = random.randint()
 import sys
 sys.stderr.write('\nBefore!\n')
 for key, value in request.session.iteritems():
     sys.stderr.write('%s=%s\n'%(key, value))
 request.session.save()
 sys.stderr.write('\nAfter!\n')
 for key, value in request.session.iteritems():
     sys.stderr.write('%s=%s\n'%(key, value))

The results on my local development machine are what a sane person would expect, but the results on the production server are what an insane person might expect.

I.e. On the production server, sometimes the results are OK, while other times the "before" and "after" values don't match up!!!
The "before" value might be right, but then the "after" value displays the previous value (from the last request).

I know it sounds like these two issues are not completely related, but the code on both machines is exactly the same! What the heck is going on???

[Matt <carsrule@…>]

Continuing on from the previous comment... or sometimes the "after" value just disappears for that 'hello' key!

[Sasha Romijn]

This is certainly curious. What settings regarding sessions have you set, and are you using exactly the same database server? And are the settings between the two boxes the same? Calling it a "local development" machine makes me think DEBUG might already be set differently.

[Matt <carsrule@…>]

Using the default session engine, which I think is the database.

Yep, both running PostgreSQL 9.1.13

Settings are exactly the same (synchronisation managed by Git) and sorry should have been more clear, I get sane/desirable results on my local machine when DEBUG is True and False (i.e. when I'm in development and production mode).

[Matt <carsrule@…>]

Upgraded both machines to Ubuntu 14.04, which included an upgrade from PostgreSQL 9.1.13 to PostgreSQL 9.3

The problem still occurred.

I just tried switching from the default database session engine to a file system session engine and the problem seems to disappear.

Will try and do some investigation on that (not sure how helpful I will be), but it definitely looks like the database session engine is at fault here.

[Matt <carsrule@…>]

I'm pretty sure this is what's going on with the database session engine:

My manual call to save the session (request.session.save()) is working properly (have confirmed this), but then another copy of a request (and hence it's copy of a session) is overwriting my manual save.

I.e. I put debug prints in the Django database session engine code and noticed my manual save (looking as it should), but then there are two more saves after my manual one which sometimes (randomly) have the session data/values before my manual save and thus overwrite my desired data/values.

I'm guessing these overwriting saves are happening in the session middleware, but will confirm this.

So the problem seems to lie with the fact that copies of a request (and hence session) are being made, as opposed to creating references/pointers to the one canonical request (and session).

[Matt <carsrule@…>]

I have pinpointed the issue to as close as I can get without spending an enormous amount of time on it.

My latest find is that the "_session_cache" attribute of the SessionBase class (parent of the database SessionStore class) is changing when the database Session object/record gets saved.

I.e. I debug printed the value and id() of the _session_cache attribute before and after the transaction/atomic database save and sometimes (randomly) there are differences (which there shouldn't be).

Since session lookups are based off the _session_cache attribute, this is the most likely cause of the problem, but I have no idea why it is occurring on my production server machine and not my development machine.

Seems like some crazy Python bug to me, because I can't see why a database save would modify a different object, but who knows.

I'm just going to use the file system session engine for now (because I can't tolerate this absurd behaviour anymore), but I do hope someone is able to shed some light.

[Tim Graham]

I am going to close this as needsinfo for now as I am not sure we'll be able to reproduce this given the details so far. If you can provide a minimal project that reproduces the issue that would be helpful.