Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#33008 closed Bug (fixed)

prefetch_related() for deleted GenericForeignKey is not consistent.

Reported by: Martin Svoboda Owned by: Martin Svoboda
Component: contrib.contenttypes Version: 3.2
Severity: Normal Keywords:
Cc: Simon Charette, Mariusz Felisiak Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

prefetch_related called for GenericForeignKey sets content_type_id and object_id to None, if the foreign object doesn't exist. This behaviour is not documented.

GenericForignKey is often used for audit records, so it can keep links to non-existing objects. Probably prefetch_related shouldn't touch original values of object_id and content_type_id and only set content_object to None.

from django.contrib.auth.models import User
from django.contrib.contenttypes.fields import GenericForeignKey
from django.contrib.contenttypes.models import ContentType
from django.db import models


class TaggedItem(models.Model):
    tag = models.SlugField()
    content_type = models.ForeignKey(ContentType, on_delete=models.CASCADE)
    object_id = models.PositiveIntegerField()
    content_object = GenericForeignKey('content_type', 'object_id')

# init data
guido = User.objects.create(username='Guido')
t = TaggedItem(content_object=guido, tag='test')
t.save()
guido.delete()

# get content_object normally
tags_1 = TaggedItem.objects.filter(tag='test')
tags_1[0].content_object  # returns None
tags_1[0].object_id  # returns 1
tags_1[0].content_type_id  # returns X

# use prefetch_related
tags_2 = TaggedItem.objects.filter(tag='test').prefetch_related("content_object")
tags_2[0].content_object  # returns None
tags_2[0].object_id  # returns None
tags_2[0].content_type_id # returns None

Change History (12)

comment:1 by Carlton Gibson, 3 years ago

Cc: Simon Charette Mariusz Felisiak added
Component: Database layer (models, ORM)contrib.contenttypes
Keywords: Documentation added
Triage Stage: UnreviewedAccepted
Type: UncategorizedCleanup/optimization

Hi Martin. I do agree that's a little surprising. I'll accept as a Documentation issue on the content types app, but I'll cc Simon and Mariusz, in case they want to take it as a bug. Thanks.

comment:2 by Carlton Gibson, 3 years ago

Summary: prefetch_related for GenericForeignKey - Object doesn't exist behaviourDocument prefetch_related for GenericForeignKey - Object doesn't exist behaviour

comment:3 by Simon Charette, 3 years ago

I didn't look at the issue in detail but I assume this is happening due to the prefetching logic performing a tags_2[0].content_object = None assignment.

How does the following code behaves?

tags_1 = TaggedItem.objects.filter(tag='test')
tags_1.content_object = None
assert tags_1.object_id is None
assert tags_1.content_type_id is None

in reply to:  3 comment:4 by Martin Svoboda, 3 years ago

Thank you, you are right, assignment tags_2[0].content_object = None also set object_id and content_type_id is to None. However I am not sure if "modification" of a source object is the correct behaviour for prefetch_related.

Replying to Simon Charette:

I didn't look at the issue in detail but I assume this is happening due to the prefetching logic performing a tags_2[0].content_object = None assignment.

How does the following code behaves?

tags_1 = TaggedItem.objects.filter(tag='test')
tags_1.content_object = None
assert tags_1.object_id is None
assert tags_1.content_type_id is None

comment:5 by Simon Charette, 3 years ago

Thanks for trying it out.

There must be a way to avoid this assignment overriding somehow as this analogous situation doesn't result in attribute loss

class UserRef(models.Model):
    user = models.ForeignKey(User, models.DO_NOTHING, null=True, db_constraint=False)

UserRef.objects.create(user_id=42)
ref = UserRef.objects.prefetch_related('user')[0]
assert ref.user is None
assert ref.user_id == 42

The distinction between the two is due to this branch where GenericForeignKey.get_prefetch_queryset sets is_descriptor to True.

I haven't tried it out but I suspect that switching is_descriptor to False instead now that GenericForeignKey has been changed to use the fields cache (bfb746f983aa741afa3709794e70f1e0ab6040b5) would address your issue and unify behaviours.

comment:6 by Martin Svoboda, 3 years ago

Thank you, your recommendation actually fix the problem. I propose the patch https://github.com/django/django/pull/14762

I found one problem with the solution. prefetch_related sets tag.content_object to None, but access to this tag.content_object actually cause extra lookup query.

comment:7 by Simon Charette, 3 years ago

Looks like the fetching of the object is due to the logic in __get__ https://github.com/django/django/blob/54a30a7a00fea6c5e3702282ade6e0238e06de3b/django/contrib/contenttypes/fields.py#L231-L245.

Not sure how feasible it would be to change it without breaking backward compatiblity though, you'd likely need to avoid passing default=None to get_cached_value and differentiate the sentinel value from None.

comment:8 by Jacob Walls, 3 years ago

Has patch: set

comment:9 by Martin Svoboda, 3 years ago

I updated the object fetching logic of __get__. It respects None as value if the instance exists in the cache. Is this solution ok? It came to me as more readable than passing different None values.

https://github.com/martinsvoboda/django/commit/7c83cec8b3bb99d9fe1822c3f82e58b373338a63#diff-76b33afd71c30afb749a0de312cfe4a4578d78abae84a9d5db06388a4b1bc454R231

comment:10 by Mariusz Felisiak, 3 years ago

Keywords: Documentation removed
Owner: changed from nobody to Martin Svoboda
Status: newassigned
Summary: Document prefetch_related for GenericForeignKey - Object doesn't exist behaviourprefetch_related() for deleted GenericForeignKey is not consistent.
Triage Stage: AcceptedReady for checkin
Type: Cleanup/optimizationBug
Version 0, edited 3 years ago by Mariusz Felisiak (next)

comment:11 by Mariusz Felisiak <felisiak.mariusz@…>, 3 years ago

Resolution: fixed
Status: assignedclosed

In cc4cb95b:

Fixed #33008 -- Fixed prefetch_related() for deleted GenericForeignKeys.

Thanks Simon Charette for the implementation idea.

comment:12 by Mariusz Felisiak <felisiak.mariusz@…>, 3 years ago

In dd8945d3:

[4.0.x] Fixed #33008 -- Fixed prefetch_related() for deleted GenericForeignKeys.

Thanks Simon Charette for the implementation idea.

Backport of cc4cb95beff0b75ec169add7e94cc481624a41e6 from main

Note: See TracTickets for help on using tickets.
Back to Top