Getting cached instance of SortedDict using db cache backend throws AttributeError
|Reported by:||Owned by:||John Huddleston|
|Component:||Core (Cache system)||Version:||master|
|Has patch:||yes||Needs documentation:||no|
|Needs tests:||no||Patch needs improvement:||no|
Using database caching, I set a non-empty SortedDict instance in the cache. When I try to retrieve it, I get an AttributeError with the message "'SortedDict' object has no attribute 'keyOrder'". Here is the code to recreate the problem:
# Using database cache backend from django.core.cache import cache from django.utils.datastructures import SortedDict value = SortedDict() value['1'] = 1 cache.set('my_dict', value) cache.get('my_dict')
I looked into the database cache backend and traced the problem to the pickling of the SortedDict instance which can be recreated like this:
import base64 import pickle from django.utils.datastructures import SortedDict value = SortedDict() value['1'] = 1 encoded = base64.encodestring(pickle.dumps(value, 2)).strip() decoded = pickle.loads(base64.decodestring(encoded))
The call to pickle.loads results in the AttributeError. If the pickling protocol is changed in dumps to 0, the call to loads works. The problem appears to be that protocol 2 calls SortedDict.__new__ instead of creating an empty SortedDict. When __new__ is called, __init__ is not being called so keyOrder is not being initialized.
If this is really unexpected behavior, the problem could be solved by changing the pickle protocol in the database caching backend from 2 to 0. It looks like the database cache backend is the only backend using pickle protocol 2 while the local memory backend is using protocol 0.
Another solution would be to subclass SortedDict.__new__ so keyOrder is always set:
def __new__(cls, *args, **kwargs): instance = super(SortedDict, cls).__new__(cls, *args, **kwargs) instance.keyOrder =  return instance
I have attached patches reflecting both of these solutions (db.py.diff and datastructures.py.diff, respectively).
Change History (6)
comment:1 Changed 8 years ago by
|Owner:||changed from nobody to John Huddleston|
|Patch needs improvement:||unset|
|Status:||new → assigned|