Prefetching a reverse GenericRelation raises AttributeError

[john]

I have some models in my app:

class User(models.Model):
    name = models.CharField()


class Project(models.Model):
    name = models.CharField()
    users = models.ManyToManyField('User', related_name='projects')


class Task(models.Model):
    name = models.CharField()
    project = models.ForeignKey('Project', on_delete=models.DO_NOTHING, related_name='tasks', null=True)

    link_entity_type = models.ForeignKey(ContentType, on_delete=models.DO_NOTHING, null=True, related_name='link_tasks')
    link_entity_id = models.IntegerField(null=True)
    link = GenericForeignKey('link_entity_type', 'link_entity_id')


class Version(models.Model):
    name = models.CharField()
    task = models.ForeignKey('Task', on_delete=models.DO_NOTHING, related_name='versions', null=True)


class Shot(models.Model):
    name = models.CharField()
    users = models.ManyToManyField('User', related_name='shots')

    link_tasks = GenericRelation(
        'Task',
        object_id_field='link_entity_id',
        content_type_field='link_entity_type'.format(field.name),
        related_query_name='link_Shot'
    )


class Asset(models.Model):
    name = models.CharField()
    users = models.ManyToManyField('User', related_name='assets')

    link_tasks = GenericRelation(
        'Task',
        object_id_field='link_entity_id',
        content_type_field='link_entity_type'.format(field.name),
        related_query_name='link_Asset'
    )

If I query Task and prefetch project and users, I will use:

query = Task.objects.all().prefetch_related('project__users')
for i in query:
    print(i.id)

This is ok, I can get the result.

But if I want to query Task and prefetch link_Shot and users

query = Task.objects.all().prefetch_related('link_Shot__users')
for i in query:
    print(i.id)

or

query = Version.objects.all().prefetch_related('task__link_Shot__users')
for i in query:
    print(i.id)

I got error:

Traceback (most recent call last):
  File "/ela/workspace/xinghuan/issues/sins/sins2server/tmp/tst_query.py", line 35, in <module>
    for i in query:
  File "/UNC/fs_user_ws.ela.com/ela/workspace/xinghuan/issues/sins/sins2server/tmp/django/db/models/query.py", line 398, in __iter__
    self._fetch_all()
  File "/UNC/fs_user_ws.ela.com/ela/workspace/xinghuan/issues/sins/sins2server/tmp/django/db/models/query.py", line 1883, in _fetch_all
    self._prefetch_related_objects()
  File "/UNC/fs_user_ws.ela.com/ela/workspace/xinghuan/issues/sins/sins2server/tmp/django/db/models/query.py", line 1273, in _prefetch_related_objects
    prefetch_related_objects(self._result_cache, *self._prefetch_related_lookups)
  File "/UNC/fs_user_ws.ela.com/ela/workspace/xinghuan/issues/sins/sins2server/tmp/django/db/models/query.py", line 2321, in prefetch_related_objects
    obj_list, additional_lookups = prefetch_one_level(
  File "/UNC/fs_user_ws.ela.com/ela/workspace/xinghuan/issues/sins/sins2server/tmp/django/db/models/query.py", line 2463, in prefetch_one_level
    ) = prefetcher.get_prefetch_queryset(instances, lookup.get_current_queryset(level))
  File "/UNC/fs_user_ws.ela.com/ela/workspace/xinghuan/issues/sins/sins2server/tmp/django/db/models/fields/related_descriptors.py", line 746, in get_prefetch_queryset
    instance = instances_dict[rel_obj_attr(rel_obj)]
  File "/UNC/fs_user_ws.ela.com/ela/workspace/xinghuan/issues/sins/sins2server/tmp/django/db/models/fields/related.py", line 750, in get_local_related_value
    return self.get_instance_value_for_fields(instance, self.local_related_fields)
  File "/UNC/fs_user_ws.ela.com/ela/workspace/xinghuan/issues/sins/sins2server/tmp/django/db/models/fields/related.py", line 772, in get_instance_value_for_fields
    ret.append(getattr(instance, field.attname))
AttributeError: 'Shot' object has no attribute 'link_entity_id'

The discussion link:
​https://forum.djangoproject.com/t/how-to-prefetch-a-genericforeignkey-with-manytomany-field/38168

I'm using Django 4.2.16 and python 3.9.
I also tried Django 5.1.5 and python 3.10 and the error is same.

[Sarah Boyce]

Thank you for the report!

Replicated with Simon's test case in the forum, which raises: AttributeError: 'Bookmark' object has no attribute 'favorite_fkey'. Did you mean: 'favorite_tags'?

tests/prefetch_related/tests.py

@@ -1243 +1243 @@
                 ],
             )
 
+    def test_reverse_gfk(self):
+        bookmark = Bookmark.objects.create()
+        TaggedItem.objects.create(content_object=bookmark, favorite=bookmark)
+        with self.assertNumQueries(1):
+            list(TaggedItem.objects.prefetch_related("favorite_bookmarks"))
+
 class MultiTableInheritanceTest(TestCase):
     @classmethod

[Simon Charette]

Changing the title as direct GenericRelation fetching works but their reverse, as defined by related_query_name, doesn't.

[Sarah Boyce <42296566+sarahboyce@…>]

In 198b301:

Fixed #36135 -- Fixed reverse GenericRelation prefetching.

The get_(local|foreign)_related_value methods of GenericRelation must be
reversed because it defines (from|to)_fields and associated related_fields
in the reversed order as it's effectively a reverse GenericForeignKey
itself.

The related value methods must also account for the fact that referenced
primary key values might be stored as a string on the model defining the
GenericForeignKey but as integer on the model defining the GenericRelation.
This is achieved by calling the to_python method of the involved content type
in get_foreign_related_value just like GenericRelatedObjectManager does.

Lastly reverse many-to-one manager's prefetch_related_querysets should use
set_cached_value instead of direct attribute assignment as direct assignment
might are disallowed on ReverseManyToOneDescriptor descriptors. This is likely
something that was missed in f5233dc (refs #32511) when the is_cached guard
was added.

Thanks 1xinghuan for the report.

[Sarah Boyce <42296566+sarahboyce@…>]

In 303c256:

[5.2.x] Fixed #36135 -- Fixed reverse GenericRelation prefetching.

The get_(local|foreign)_related_value methods of GenericRelation must be
reversed because it defines (from|to)_fields and associated related_fields
in the reversed order as it's effectively a reverse GenericForeignKey
itself.

The related value methods must also account for the fact that referenced
primary key values might be stored as a string on the model defining the
GenericForeignKey but as integer on the model defining the GenericRelation.
This is achieved by calling the to_python method of the involved content type
in get_foreign_related_value just like GenericRelatedObjectManager does.

Lastly reverse many-to-one manager's prefetch_related_querysets should use
set_cached_value instead of direct attribute assignment as direct assignment
might are disallowed on ReverseManyToOneDescriptor descriptors. This is likely
something that was missed in f5233dc (refs #32511) when the is_cached guard
was added.

Thanks 1xinghuan for the report.

Backport of 198b30168d4e94af42e0dc7967bd3259b5c5790b from main.