ValueError when FileBasedCache is used in Python 3.7

[mastizada]

Using FileBasedCache in Django 2.2.12 with Python 3.7.3 is giving ValueError: unsupported pickle protocol: 5

Checking the highest protocol version for pickle using pickle.HIGHEST_PROTOCOL is giving 4, so the protocol 5 is not available.

Line that is causing the error: filebased.py in _is_expired at line 146, exp = pickle.load(f).

Using Python 3.8.0 solves the problem.

[Hasan Ramezani]

​Django 2.2 doesn't support Python2.7

[Carlton Gibson]

Django 1.11 was the last version of Django to support Python 2.7

[Simon Charette]

You likely serialized data using Python 3 and are trying to deserialize it using Python 2. Django 2.2 also doesn't support Python 2 at all.

[mastizada]

Indicated the wrong python version

[mastizada]

Replying to Simon Charette:

You likely serialized data using Python 3 and are trying to deserialize it using Python 2. Django 2.2 also doesn't support Python 2 at all.

I'm so sorry, I wrote python 2.7 instead of 3.7.

Project was working before, I updated the Django version from 2.2.9 to 2.2.12 and then started to get this error.

[Simon Charette]

Python 3.8 introduced ​pickle protocol 5 and pointed pickle.HIGHEST_PROTOCOL at it. You most likely ran your project on Python 3.8 and stored things in a cache using the FileBasedCache backend and then tried running your project on Python 3.7.

If you're in process of migrating from 3.7 to 3.8 you'll likely want to subclass FileBasedCache and set its pickle_protocol to 4 and use it as your backend during the transition period. When you've fully migrated to Python 3.8 you can swap back to FileBasedCache.

I had to do similar manipulations when moving projects from Python 2 to 3 which makes me wonder if we should add a CACHE_PICKLE_PROTOCOL setting or allow to specify it via CACHES['OPTIONS'] to avoid having to subclassing backends when such a transition is required. This is likely to be more of an issue as more and more people migrate to 3.8.

[mastizada]

Replying to Simon Charette:

If you're in the process of migrating from 3.7 to 3.8 you'll likely want to subclass FileBasedCache and set its pickle_protocol to 4 and use it as your backend during the transition period. When you've fully migrated to Python 3.8 you can swap back to FileBasedCache.

I also was thinking about the cache, that is why I deleted everything in the cache folder (/tmp/django/) but that didn't solve the error, it happened again at the second load of the page.

I think having ability to set the pickle protocol version as an option would benefit too.