Opened 9 years ago

Closed 3 years ago

#23895 closed Bug (fixed)

ResolverMatch with some views is not pickleable

Reported by: Keryn Knight Owned by: zatro
Component: Core (URLs) Version: 4.0
Severity: Normal Keywords:
Cc: django@… Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

given something like the following:

def my_fbv(request):
    return HttpResponse('yay')

# urls.py
urlpatterns = [
    url('whatever', my_fbv, name='my_fbv'),
]

It is possible to do the following:

from django.core.urlresolvers import resolve
from pickle import dumps, loads
loads(dumps(resolve('whatever')))

and end up with a working ResolverMatch

However, given a Class Based View (mapped to a urlconf via MyView.as_view()), or something like contrib.admin, you get something like the following:

dumps(resolve('/admin/myapp/'))
[...]
# for the admin ...
PicklingError: Can't pickle <function app_index at 0x109f05de8>: it's not found as django.contrib.admin.sites.app_index
# for a CBV:
PicklingError: Can't pickle <function Homepage at 0x109f16b90>: it's not the same object as myapp.views.Homepage

Both of which are raised by pickle's save_global(self, obj, name, pack) which recognises that it's a module+name combo (thus should be in scope) but isn't the same object in identity (if x is not y)

Ordinarily, this is not a problem, but resolver_match is set onto a request, and I'm using the django.test.client.Client with multiprocessing, which requires the ability to pickle data to send back and forth, and evidently somewhere within the TemplateResponses I'm dealing with, the request is being serialised (if I had to guess -- probably in the context) and is sometimes failing (depending on the type of view mounted)

Ideally, every ResolverMatch should be serialisable, or none should be, instead of the current situation where the project's urlconf may denote success or failure.

Change History (11)

comment:1 by Tim Graham, 9 years ago

Triage Stage: UnreviewedAccepted

comment:2 by Tom Christie, 9 years ago

Haven't had a chance to confirm this much but appears that on 1.8-alpha this is consistently an issue, making responses non-cachable with CacheMiddleware. Alternative take would be for HttpResponse to take a more structured approach to caching - rather than simply dumping all attributes to pickle, just store the basics that allow a fresh HttpResponse with correct content, status code and headers to be reconstructed. (Eg. we shouldn't expect ResolverMatch instances to be picklable, but we *should* be constraining what we pickle for responses)

comment:3 by zatro, 3 years ago

Owner: changed from nobody to zatro
Status: newassigned

comment:4 by zatro, 3 years ago

Has patch: set
Version: 1.74.0

I've done a fix for this but it might not be the ideal solution. Thoughts? If it's okay, please let me know anything the PR is missing. PR

comment:5 by Mariusz Felisiak, 3 years ago

Patch needs improvement: set

comment:6 by zatro, 3 years ago

Patch needs improvement: unset

I've now made the changes discussed and squashed the commits.

comment:7 by Mariusz Felisiak, 3 years ago

Patch needs improvement: set

comment:8 by zatro, 3 years ago

Patch needs improvement: unset

Changes from latest review have now been made.

comment:9 by zatro, 3 years ago

Made a new ticket #32969 for improving pickling of HTTPResponses.

comment:10 by Mariusz Felisiak, 3 years ago

Triage Stage: AcceptedReady for checkin

comment:11 by GitHub <noreply@…>, 3 years ago

Resolution: fixed
Status: assignedclosed

In 4c6a6d5a:

Fixed #23895 -- Prevented pickling of ResolverMatch.

Pickling a ResolverMatch did not work correctly in many cases,
especially with CBVs and URLResolvers in the list of tried URL paths.

Note: See TracTickets for help on using tickets.
Back to Top