Opened 11 years ago
Closed 4 years ago
#22382 closed Bug (duplicate)
ManyRelatedManager's get_prefetch_queryset doesn't validate the prefetch types
Reported by: | Owned by: | Marc Tamlyn | |
---|---|---|---|
Component: | Database layer (models, ORM) | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Accepted | |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
the return value for get_prefetch_queryset
includes two lambda functions, one for setting the resulting tuple as a key in the rel_obj_cache
and another for getting it back later.
The one which is used for retrieving the data uses getattr
while the other pulls the data from the _prefetch_related_val_...
attribute returned as part of the QuerySet.extra
call. The prefetched values do not necessarily correlate to the intended types, however, and so values can end up mismatched.
By way of example, and how I discovered this, django-uuidfield returns a StringUUID
in it's to_python method, which is what ends up being returned as instance_attr - something like (UUID('7d917781c54e4fdfa551a693c3782380'),)
but the rel_obj_attr doesn't take into account the to_python
for the relation, and so returns (u'7d917781c54e4fdfa551a693c3782380',)
- the naive strings from the database.
Thus, the key (u'7d917781c54e4fdfa551a693c3782380',)
never exists in the rel_obj_cache
, and Django assumes everything went well (because it just sets a default of []
). Future queries to the relation (x.relation.all()
) will yield nothing - they won't trigger a query because they've been prefetched, but they won't be populated because of the type mismatch in the dictionary keys.
Arguably the problem exists downstream in django-uuidfield
, but the problem *also* exists in Django, and may exist as an edge-case in other third party fields - I've marked it as master, but the type-mismatch issue exists from 1.4 (attrgetter version) to 1.6 (list comprehension version) and looks to still be there in master (generator expression version)
The fix that seems to work for me is transforming:
lambda result: tuple(getattr(result, '_prefetch_related_val_%s' % f.attname) for f in fk.local_related_fields),
into:
lambda result: tuple(f.rel.get_related_field().to_python(getattr(result, '_prefetch_related_val_%s' % f.attname)) for f in fk.local_related_fields),
I'm not sure if any of the equivalent methods on other classes would be affected.
Change History (4)
comment:1 by , 11 years ago
Triage Stage: | Unreviewed → Accepted |
---|
comment:2 by , 10 years ago
Owner: | changed from | to
---|---|
Status: | new → assigned |
I think this is a duplicate of https://code.djangoproject.com/ticket/24912
and got fixed by https://github.com/django/django/commit/c58755d8757d6d6ad1ab2c52a631aff1b9bae2da.