SESSION_SAVE_EVERY_REQUEST = True does not handle parallel requests well if some scenarios

[Michael]

In my project, there are some methods on a custom user model that require the request to calculate certain values. This simple middleware does the trick:

class AttachRequestToUserMiddleware:
    """Adds the request to the user object, so session information can be looked
    up by the custom user model.

    Must come after django.contrib.auth.middleware.AuthenticationMiddleware which adds the user"""

    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        request.user.request = request
        return self.get_response(request)

In production, when there are multiple workers running parrallel by uWSGI, if one has SESSION_SAVE_EVERY_REQUEST = True, then if one makes async requests from JavaScript (e.g. say a Service Worker caching pages on install), then the way it saves/retrieves sessions on every request fails in many spectacular ways.

Here are some example tracebacks of the many errors raised:

Django Version:	4.0.1
Python Version:	3.8.5

Exception Type:	ProgrammingError
Exception Value:	
no results to fetch
Exception Location:	/home/michael/.venv/project/lib/python3.8/site-packages/django/db/utils.py, line 97, in inner
/home/michael/.venv/project/lib/python3.8/site-packages/django/db/utils.py, line 97, in inner
                return func(*args, **kwargs)


Exception Type:	RuntimeError
Exception Value:	
generator raised StopIteration
/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/auth/__init__.py, line 60, in _get_user_session_key
return get_user_model()._meta.pk.to_python(request.session[SESSION_KEY])


Exception Type:	MultipleObjectsReturned
Exception Value:	
get() returned more than one Session -- it returned 2!
/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/base.py, line 180, in _get_session
            return self._session_cache
/home/michael/.venv/project/lib/python3.8/site-packages/django/core/handlers/exception.py, line 47, in inner
                response = get_response(request)
./core/accounts/middleware.py, line 33, in __call__
        request.user.request = request
/home/michael/.venv/project/lib/python3.8/site-packages/django/utils/functional.py, line 278, in __setattr__
                self._setup()
/home/michael/.venv/project/lib/python3.8/site-packages/django/utils/functional.py, line 384, in _setup
        self._wrapped = self._setupfunc()
/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/auth/middleware.py, line 25, in <lambda>
        request.user = SimpleLazyObject(lambda: get_user(request))
/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/auth/middleware.py, line 11, in get_user
        request._cached_user = auth.get_user(request)
/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/auth/__init__.py, line 177, in get_user
        user_id = _get_user_session_key(request)
/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/auth/__init__.py, line 60, in _get_user_session_key
    return get_user_model()._meta.pk.to_python(request.session[SESSION_KEY])
/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/base.py, line 50, in __getitem__
        return self._session[key]
/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/base.py, line 185, in _get_session
                self._session_cache = self.load()
/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/db.py, line 43, in load
        s = self._get_session_from_db()
/home/michael/.venv/project/lib/python3.8/site-packages/django/contrib/sessions/backends/db.py, line 32, in _get_session_from_db
            return self.model.objects.get(
/home/michael/.venv/project/lib/python3.8/site-packages/django/db/models/manager.py, line 85, in manager_method
                return getattr(self.get_queryset(), name)(*args, **kwargs)
/home/michael/.venv/project/lib/python3.8/site-packages/django/db/models/query.py, line 443, in get
        raise self.model.MultipleObjectsReturned(

[Tim Graham]

See TicketClosingReasons/UseSupportChannels for ways to get help if you cannot debug the issue yourself. Please only use this ticket tracker for reporting confirmed bugs. Thanks!

[Michael]

Replying to Tim Graham:

See TicketClosingReasons/UseSupportChannels for ways to get help if you cannot debug the issue yourself. Please only use this ticket tracker for reporting confirmed bugs. Thanks!

I actually opened a stack overflow question about a week ago and got no responses, and I think it's due to the fact you can't debug a dictionary lookup, it seems like quite an atomic action. I was hestitant on creating this issue, but after giving it some thought, I suspected it must be a bug not the code, since the code is basically set a key in the session, then read the key.

[Michael]

Replying to Tim Graham:

See TicketClosingReasons/UseSupportChannels for ways to get help if you cannot debug the issue yourself. Please only use this ticket tracker for reporting confirmed bugs. Thanks!

Hi, I have done some more digging with some help: ​https://forum.djangoproject.com/t/should-it-be-impossible-for-a-session-dict-to-return-more-than-one-value-for-a-key/12298/13

I think the issue is this: if it receives multiple requests at the same time (I have a service worker that requests many pages at installation so it can cache them), and if SESSION_SAVE_EVERY_REQUEST = True, then the way it handles session updates it just falls over in many spectacular ways.
My for now my fix is to set SESSION_COOKIE_AGE to over a year, and have SESSION_SAVE_EVERY_REQUEST = False