#16563 closed Bug (fixed)
Error pickling request.user
Reported by: | Owned by: | nobody | |
---|---|---|---|
Component: | contrib.auth | Version: | 1.3 |
Severity: | Release blocker | Keywords: | |
Cc: | Torsten Bronger, kmike84@… | Triage Stage: | Accepted |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description (last modified by )
trying to pickle a request.user in trunk raises: TypeError, can't pickle function objects
Looking it shows that request.user is a SimpleLazyObject and it is a <lambda>, so it can not be pickled.
try: import cPickle as pickle except: import pickle def some_view(request): pickle.dumps(request.user) # raise type error …
Attachments (2)
Change History (25)
comment:1 by , 13 years ago
Resolution: | → wontfix |
---|---|
Status: | new → closed |
comment:2 by , 13 years ago
milestone: | → 1.4 |
---|---|
Resolution: | wontfix |
Status: | closed → reopened |
Triage Stage: | Unreviewed → Design decision needed |
I'm going to reopen for now due to the fact this is a regression in 1.4.
Previously, this was possible:
def my_view(request): obj = MyObj(user=request.user, amount=1) request.session['unsaved_obj'] = obj # ...
Now request.user has been changed to a SimpleLazyObject
rather than changing the request class to actually lazily return the true user object, this will choke.
Maybe this is an acceptable regression, but it's something I tripped up on in a project recently.
comment:3 by , 13 years ago
Severity: | Normal → Release blocker |
---|
If this is a regression, then it should be tracked as a release blocker.
comment:4 by , 13 years ago
Description: | modified (diff) |
---|
comment:5 by , 13 years ago
Cc: | added |
---|
comment:6 by , 13 years ago
Triage Stage: | Design decision needed → Accepted |
---|
comment:8 by , 13 years ago
Same problem with Django 1.4 (trunk), but in Django 1.3 all fine
from django.core.cache import cache car = Car.objects.get(pk=8809) car.name = 'test' car.save() cache.set('test', request.user, 1)
Raise error:
can't pickle function objects
C:\Python\lib\site-packages\django\core\cache\backends\locmem.py in set
self._set(key, pickle.dumps(value), timeout)
C:\Python\lib\copy_reg.py in _reduce_ex
raise TypeError, "can't pickle %s objects" % base.name
comment:9 by , 13 years ago
Is this regression related to the change to using pickle.HIGHEST_PROTOCOL
throughout Django? The memcached backend used that, but locmem didn't. Does that mean this isn't a regression after all, and depended on the non-canonical behavior of locmem using the lowest pickle protocol?
comment:11 by , 13 years ago
Owner: | changed from | to
---|---|
Status: | reopened → new |
comment:12 by , 13 years ago
Owner: | changed from | to
---|
comment:13 by , 13 years ago
I wouldn't call this a regression unless Django promised you could do it in 1.3, at least implicitly. Not every behavior change is a regression.
follow-up: 18 comment:14 by , 13 years ago
FYI, the current code that makes request.user a SimpleLazyObject is from changeset [16305], which was a re-working of [16297], to fix #15929.
A comment cleaned up along the way said:
# If we access request.user, request.session is accessed, which results in # 'Vary: Cookie' being sent in every request that uses this context # processor, which can easily be every request on a site if # TEMPLATE_CONTEXT_PROCESSORS has this context processor added. This kills # the ability to cache. So, we carefully ensure these attributes are lazy.
which seems like a pretty good reason for making request.user lazy.
Would it be possible to fix this instead by fixing the chain of events somewhere else? e.g. should any access of request.session result in setting the Vary: Cookie header?
follow-up: 16 comment:15 by , 13 years ago
As a work around, can't you just pickle a real User model object instead of request.user? E.g., the dumb way to do this might look something like:
def some_view(request): pickle.dumps(User.objects.get(pk=request.user.pk))
follow-up: 17 comment:16 by , 13 years ago
Replying to tobias:
As a work around, can't you just pickle a real User model object instead of request.user?
That would still require a code change when upgrading to 1.4, right?
comment:17 by , 13 years ago
Replying to poirier:
Replying to tobias:
As a work around, can't you just pickle a real User model object instead of request.user?
That would still require a code change when upgrading to 1.4, right?
Yes, I meant it as a complement to your original argument. As you said, being able to pickle request.user directly was never documented, so whether or not it's a release blocking regression is debatable.
comment:18 by , 13 years ago
Replying to poirier:
Would it be possible to fix this instead by fixing the chain of events somewhere else?
No, I don't think so.
e.g. should any access of request.session result in setting the Vary: Cookie header?
Yes, it should. Any access of the session means the response you are generating is almost certainly dependent in some way on values in the session, which means serving that same response as a cached response to other users would be at best wrong, and at worst a security issue. This applies even more strongly, if anything, to accessing request.user
in particular. So it's quite important that request.user
remain lazy, and that accessing it trigger Vary: Cookie
on the response.
comment:19 by , 13 years ago
Cc: | added |
---|
Are there obstacles for making LazyObject instances picklable? E.g.:
class LazyObject(object): # ... def __getstate__(self): if self._wrapped is None: self._setup() return self._wrapped def __setstate__(self, state): self._wrapped = state
comment:20 by , 13 years ago
@kmike: That doesn't quite work, but it is close. My attached patch works in some cases, if someone can verify that it fixes the issue with a real User instance in environment showing the bug, I will commit it.
I added the code to SimpleLazyObject
not LazyObject
, to minimize impact with other subclasses of LazyObject
. SimpleLazyObject
is not designed to be subclassed really - you are supposed to use it just by passing a callable to the constructor.
There is one potential backwards compatibility concern here: if you had used a SimpleLazyObject
with a callable that was itself pickleable, it would previously have been possible to pickle that SimpleLazyObject
without the patch. Potentially, the object could have been pickled in the 'lazy' state, without the callable having been called.
However, I've tested this, and it seems that - for whatever reason - calling pickle.dumps
causes the the callable to be called. Therefore, the object is not serialised in a 'lazy' state - it is always 'evaluated'. And this is just the same as it is with the patch, except I have made it explicit. The only difference now is that SimpleLazyObject._setupfunc
is never available now after unpickling, but that shouldn't be a problem since it is never used.
I'm therefore fairly confident that this patch shouldn't cause other problems.
comment:21 by , 13 years ago
OK, this has turned out to be rather tricky.
The previous patch only works for objects that have no state. The basic problem is that due to proxying the __class__
attribute, the pickling machinery is confused (I think), and so it stores a reference to the class of the wrapped object, rather than SimpleLazyObject
.
So, we need to define the __reduce__
method for full customization, rather than the simpler __getstate__
method. However, for some reason, the presence of the __class__
attribute proxying causes this to fail - if you define __reduce__
on SimpleLazyObject
, it is never called when pickling unless you remove the __class__
hacking.
This takes us back to using __getstate__
(which, for some reason, doesn't suffer from the same problem as __reduce__
). But now we have to cooperate with the fact that pickling will skip SimpleLazyObject
and store a reference to the wrapped class, and define __getstate__
so that it returns the state from the wrapped object. This appears to work:
def __getstate__(self): if self._wrapped is empty: self._setup() return self._wrapped.__dict__
The result is that on unpickling, the SimpleLazyObject
wrapper disappears entirely, which should be OK.
I still get failure if SimpleLazyObject
is wrapping a builtin, and nothing but removing the __class__
proxying appears to be able to fix that. But that bug has always been there, and this change doesn't affect it.
I've confirmed that with this patch you can pickle request.user
, and on unpickling get an object that compares equal to the original, so I'm therefore committing my improved patch.
Some things aren't pickleable. It's a known limitation at the Python level, and is unlikely to change. Is Django itself trying to do this somewhere? Is there an extremely good reason this should be possible? I'm closing this as wontfix. Please feel free to reply if you've got good answers to these questions.