Prefetch_related fails with GenericRelation

[okke@…]

While greatly anticipating prefetch_related, I am sorry to say that bug #17838 is not yet entirely fixed. I am running into empty result sets when I perform a prefetch_related.

I have a GenericRelation on my model to Comments. My model uses an integer primary key. This combination makes the prefetch_related return empty embedded result sets.

In the latest django, django/db/models/query.py, line 1791, the _prefetched_objects_cache is accessed with (possibly) an integer as value. However, the _prefetched_objects_cache is always populated with unicode values (lines 1770 - 1774)

The quick fix is to replace (line 1791)

            obj._prefetched_objects_cache[cache_name] = qs

with

            obj._prefetched_objects_cache[unicode(cache_name)] = qs

Can someone create a patch for this, and write a test to see if the queryset is actually populated with related items?

[anonymous]

I'm sorry, I made a mistake in the line in which the error is.

It's line 1778:

        vals = rel_obj_cache.get(instance_attr_val, [])

which should be

        vals = rel_obj_cache.get(unicode(instance_attr_val), [])

[Artem Skoretskiy]

I got the same bug -- see #18200 . The fix in #comment:1 fixed the issue for me

[stephanejais]

There's more to patch if you want "relations of relations" to also work:

> diff /Users/stephh/.virtualenv/azap/lib/python2.7/site-packages/django/db/models/query.py django/db/models/query.py
1773,1774c1773,1774
<             rel_obj_cache[rel_attr_val] = []
<         rel_obj_cache[rel_attr_val].append(rel_obj)
---
>             rel_obj_cache[unicode(rel_attr_val)] = []
>         rel_obj_cache[unicode(rel_attr_val)].append(rel_obj)
1778c1778
<         vals = rel_obj_cache.get(instance_attr_val, [])
---
>         vals = rel_obj_cache.get(unicode(instance_attr_val), [])

[Luke Plant]

I think at this point a test demonstrating the problem would be the most useful thing. Blindly converting things to unicode seems like altogether the wrong approach. If there is any type conversion to be done, it is surely something that the model fields will dictate.

[carmandrew@…]

I ran into the same issue. There is a bug with the way that prefetch_related is handling GenericRelations where the GenericForeignKey's object_id is a different type than the primary key of the model it's pointing to. I've attached a test case.

[Luke Plant <L.Plant.98@…>]

In 4c4d08502cbef6e0ed85470b2a5aade7fafa099f:

Fixed #17991 - prefetch_related fails with GenericRelation and varchar ID field

Thanks to okke@… for the report, and carmandrew@… for the tests.

[Luke Plant <L.Plant.98@…>]

In d7d7ad2881b66d2c6be45ab0b055f492d3b589e9:

[1.5.x] Fixed #17991 - prefetch_related fails with GenericRelation and varchar ID field

Thanks to okke@… for the report, and carmandrew@… for the tests.

Backport of ccd14ff25b7642678bf3c9ed8a12643f04853144 from master