KeyError in django-sites caching (race condition?)

[Dominik George]

Django Version: 1.10.8
Python Version: 3.5.3

Relevant backtrace:

File
"/srv/www/teckids-website/venv/lib/python3.5/site-packages/django/contrib/sites/models.py"
in _get_site_by_id
  37.         return SITE_CACHE[site_id]

File "/srv/www/teckids-website/venv/lib/python3.5/site-packages/multisite/hacks.py" in
__getitem__
  124.             raise KeyError(key)

In theory, there cannot be a KeyError there, unless the cache is cleared while the method is running. The code does not seem to be thread-safe.

[Dominik George]

After a discussion at Chemnitzer Linux-Tage with some Django people, we agreed it is a bug, but maybe related to Apache mpm_prefork's multi-threading behaviour.

We reduced the threads in each process to 1 in the mod_wsgi config, but the error still occured after that.

[Tim Graham]

I'm not sure. The fact that the traceback involves multisite/hacks.py (which is not part of Django) looks a bit suspicious.

[Dominik George]

Yeah. I am not saying the CMS isn't part of the problem, but I don't need a backtrace to tell that the code in Django is neither thread-safe nor otherwise good practice ☺.

[Tim Graham]

I'm skeptical because I'd think that if this were a common issue, we'd have seen it reported before now.

I don't know what can be done. Are you able to offer a patch?

[Tim Graham]

It might be a duplicate of #15894?

[Dominik George]

Nope, it's not a duplicate.