ResolverMatch with some views is not pickleable

[Keryn Knight]

given something like the following:

def my_fbv(request):
    return HttpResponse('yay')

# urls.py
urlpatterns = [
    url('whatever', my_fbv, name='my_fbv'),
]

It is possible to do the following:

from django.core.urlresolvers import resolve
from pickle import dumps, loads
loads(dumps(resolve('whatever')))

and end up with a working ResolverMatch

However, given a Class Based View (mapped to a urlconf via MyView.as_view()), or something like contrib.admin, you get something like the following:

dumps(resolve('/admin/myapp/'))
[...]
# for the admin ...
PicklingError: Can't pickle <function app_index at 0x109f05de8>: it's not found as django.contrib.admin.sites.app_index
# for a CBV:
PicklingError: Can't pickle <function Homepage at 0x109f16b90>: it's not the same object as myapp.views.Homepage

Both of which are raised by pickle's save_global(self, obj, name, pack) which recognises that it's a module+name combo (thus should be in scope) but isn't the same object in identity (if x is not y)

Ordinarily, this is not a problem, but resolver_match is set onto a request, and I'm using the django.test.client.Client with multiprocessing, which requires the ability to pickle data to send back and forth, and evidently somewhere within the TemplateResponses I'm dealing with, the request is being serialised (if I had to guess -- probably in the context) and is sometimes failing (depending on the type of view mounted)

Ideally, every ResolverMatch should be serialisable, or none should be, instead of the current situation where the project's urlconf may denote success or failure.

[Tom Christie]

Haven't had a chance to confirm this much but appears that on 1.8-alpha this is consistently an issue, making responses non-cachable with CacheMiddleware. Alternative take would be for HttpResponse to take a more structured approach to caching - rather than simply dumping all attributes to pickle, just store the basics that allow a fresh HttpResponse with correct content, status code and headers to be reconstructed. (Eg. we shouldn't expect ResolverMatch instances to be picklable, but we *should* be constraining what we pickle for responses)

[zatro]

I've done a fix for this but it might not be the ideal solution. Thoughts? If it's okay, please let me know anything the PR is missing. ​PR

[zatro]

I've now made the changes discussed and squashed the commits.

[zatro]

Changes from latest review have now been made.

[zatro]

Made a new ticket #32969 for improving pickling of HTTPResponses.

[GitHub <noreply@…>]

In 4c6a6d5a:

Fixed #23895 -- Prevented pickling of ResolverMatch.

Pickling a ResolverMatch did not work correctly in many cases,
especially with CBVs and URLResolvers in the list of tried URL paths.