id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
20493	Add a warning that objects may not be picklable across Django versions	Catalin Iacob	nobody	"I just upgraded from 1.4 to 1.5 and this took a while to track.

If one has a cache that persists across a Django version change (filesystem cache, memcached that is not restarted), when executing any view decorated with @cache_page unpickling the HttpResponse from the cache leads to strange errors. For example, when upgrading from 1.4 to 1.5 I get:

{{{
[24/May/2013 15:53:42] ""GET / HTTP/1.1"" 200 5
Traceback (most recent call last):
  File ""/usr/lib64/python2.7/wsgiref/handlers.py"", line 86, in run
    self.finish_response()
  File ""/usr/lib64/python2.7/wsgiref/handlers.py"", line 129, in finish_response
    self.close()
  File ""/usr/lib64/python2.7/wsgiref/simple_server.py"", line 36, in close
    SimpleHandler.close(self)
  File ""/usr/lib64/python2.7/wsgiref/handlers.py"", line 257, in close
    self.result.close()
  File ""/home/catalin/hacking/envs/cached-httpresponse-across-versions/lib/python2.7/site-packages/django/http/response.py"", line 231, in close
    for closable in self._closable_objects:
AttributeError: 'HttpResponse' object has no attribute '_closable_objects'
[24/May/2013 15:53:42] ""GET / HTTP/1.1"" 500 59
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 49852)
Traceback (most recent call last):
  File ""/usr/lib64/python2.7/SocketServer.py"", line 582, in process_request_thread
    self.finish_request(request, client_address)
  File ""/usr/lib64/python2.7/SocketServer.py"", line 323, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File ""/home/catalin/hacking/envs/cached-httpresponse-across-versions/lib/python2.7/site-packages/django/core/servers/basehttp.py"", line 150, in __init__
    super(WSGIRequestHandler, self).__init__(*args, **kwargs)
  File ""/usr/lib64/python2.7/SocketServer.py"", line 638, in __init__
    self.handle()
  File ""/usr/lib64/python2.7/wsgiref/simple_server.py"", line 124, in handle
    handler.run(self.server.get_app())
  File ""/usr/lib64/python2.7/wsgiref/handlers.py"", line 89, in run
    self.handle_error()
  File ""/usr/lib64/python2.7/wsgiref/handlers.py"", line 304, in handle_error
    self.finish_response()
  File ""/usr/lib64/python2.7/wsgiref/handlers.py"", line 127, in finish_response
    self.write(data)
  File ""/usr/lib64/python2.7/wsgiref/handlers.py"", line 210, in write
    self.send_headers()
  File ""/usr/lib64/python2.7/wsgiref/handlers.py"", line 267, in send_headers
    if not self.origin_server or self.client_is_modern():
  File ""/usr/lib64/python2.7/wsgiref/handlers.py"", line 280, in client_is_modern
    return self.environ['SERVER_PROTOCOL'].upper() != 'HTTP/0.9'
TypeError: 'NoneType' object has no attribute '__getitem__'
----------------------------------------
}}}

This was at first quite perplexing because I checked response.py and {{{self._closable_objects}}} gets assigned in {{{HttpResponseBase.__init__}}} which gets called by {{{HttpResponse.__init__}}} so it seemed impossible for an HttpResponse instance not to have the attribute. Unpickling doesn't normally call {{{__init__}}} but this is not at all obvious or easy to track.

There are similar issues but with a bit more clear stacktrace if downgrading from 1.5 to 1.4:
{{{
Traceback:
File ""/home/catalin/hacking/envs/cached-httpresponse-across-versions/lib/python2.7/site-packages/django/core/handlers/base.py"" in get_response
  89.                     response = middleware_method(request)
File ""/home/catalin/hacking/envs/cached-httpresponse-across-versions/lib/python2.7/site-packages/django/middleware/cache.py"" in process_request
  147.         response = self.cache.get(cache_key, None)
File ""/home/catalin/hacking/envs/cached-httpresponse-across-versions/lib/python2.7/site-packages/django/core/cache/backends/filebased.py"" in get
  41.                     return pickle.load(f)

Exception Type: ImportError at /
Exception Value: No module named response
}}}

I'm surprised nobody seems to have run into this so far, googling for AttributeError on _closable_objects and searching the other bugs didn't show anything. I would imagine it's reasonably common to upgrade without restarting memcached or to have a filesystem based cache on a development machine where you don't bother to install memcached. And this happens on '''every''' view with @cache_page.

I don't know what a good solution would be. I can imagine Django doesn't want to guarantee pickle/unpickle round trips across versions as those can be hard to maintain but I think there should at least be a warning in the upgrade notes (of every version): when upgrading Django you need to delete your cache."	Bug	closed	Documentation	1.5	Normal	fixed		iacobcatalin@…	Accepted	1	0	0	0	0	0