Code

Opened 15 months ago

Closed 15 months ago

Last modified 15 months ago

#19602 closed Bug (worksforme)

Select_related filters objects with no related member

Reported by: natmaster@… Owned by: aaugustin
Component: Database layer (models, ORM) Version: 1.5-beta-1
Severity: Release blocker Keywords: select_related
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

In Django <=1.4 if you used select_related and the related object doesn't exist, it would be filled in as None, but the original object would still show up in the results.

In Django 1.5, not having a related object when using select_related results in the original object not existing when calling get() or indexing it.

Note that the SQL is still the proper LEFT OUTER JOIN. Furthermore, a count() will result in the proper expected number of objects.

Attachments (0)

Change History (11)

comment:1 Changed 15 months ago by russellm

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Resolution set to worksforme
  • Severity changed from Normal to Release blocker
  • Status changed from new to closed
  • Triage Stage changed from Unreviewed to Accepted

This is a concerning report, since it describes a backwards incompatibility. However, I can't reproduce the problem you describe.

Using:

class OwnedObject(models.Model):
    owner = models.ForeignKey(User, null=True, blank=True)
    name = models.CharField(max_length=100)

and

>>> user1 = MyUser.objects.get(email='user1@example.com')
>>> user2 = MyUser.objects.get(email='user2@example.com')
>>> obj1 = OwnedObject.objects.create(name='obj1', owner=user1)
>>> obj2 = OwnedObject.objects.create(name='obj2', owner=user2)
>>> obj3 = OwnedObject.objects.create(name='obj2')
>>> OwnedObject.objects.select_related('owner')

the last statement returns 3 objects, as expected, even though obj3.owner doesn't exist. This is using git revision ba50d3e05b.

Closing as worksforme; please reopen if you can provide specific instructions for reproducing the problem you describe.

comment:2 Changed 15 months ago by ntucker

Isn't ba50d3e05b the 1.6.x line? AKA from the master branch?

Also, are you sure you inspected the object directly, and not just executed count()

comment:3 Changed 15 months ago by ntucker

I think this might have been my mistake, let me investigate.

comment:4 Changed 15 months ago by ntucker

  • Resolution changed from worksforme to invalid

It appears I was confusing this new behavior

https://github.com/django/django/blob/stable/1.5.x/django/db/models/fields/related.py
line 278

if rel_obj is None:

raise self.related.model.DoesNotExist

With the object not being retrieved, because I was accessing the related object after getting the first object.

This doesn't appear to be documented here https://docs.djangoproject.com/en/dev/releases/1.5/#backwards-incompatible-changes-in-1-5 though, so that might need to change.

I am curious as to why this change even took place. None representing null makes more sense to me.

comment:5 Changed 15 months ago by ntucker

Actually, this might be a bug still...because the behavior is different when select_related isn't used.

Tell me if you expect this behavior:
At the end of your example add

obj = OwnedObject.objects.select_related('owner').get(name=obj2, owner__isnull=True)
print(obj.owner)

That should throw DoesNotExist, which it does not do if you remove the select_related. Is this expected (and new in 1.5) behavior?

comment:6 Changed 15 months ago by aaugustin

  • Resolution invalid deleted
  • Status changed from closed to new

This ticket revolves around my fixes in the related object caching code.

I fixed a bunch of cases where adding select/prefetch_related didn't eliminate subsequent queries (specifically when there was no reverse related object) or changed the behavior (return None instead of DoesNotExist).

If I understand correctly the comment above there's still an instance of the second class of problems.

I'll take a look when I have access to a computer.

comment:7 Changed 15 months ago by aaugustin

  • Owner changed from nobody to aaugustin
  • Status changed from new to assigned

comment:8 Changed 15 months ago by aaugustin

  • Resolution set to worksforme
  • Status changed from assigned to closed

When a nullable foreign key is null, accessing the related object returns None. That's what you're doing in comment 5.

However, when you traverse a one-to-one relation backwards (eg. replace ForeignKey by OneToOneField in the models above and do user.ownedobject), a DoesNotExist exception is raised if no matching parent object is found. OneToOneField is asymmetrical in SQL and Django's APIs reflect it: the lack of a related object is expressed by None in the "forwards" direction and by DoesNotExist is the "backwards" direction.

In comment 4, you're quoting the code of SingleRelatedObjectDescriptor, which implements reverse lookup through one-to-one relations. (I know from experience it's very hard to follow which side of the relation the code is dealing with in this area.)

Before Django 1.5, adding select_related could change the behavior: see #13839. You were relying on this bug. By definition, bugfixes are backwards-incompatible — they change the behavior of Django — but we don't document them all in the release notes.

However, if another core dev thinks this one is sufficiently important to make an exception, I'll write a paragraph...

comment:9 follow-up: Changed 15 months ago by ntucker

If it is supposed to be throwing DoesNotExist in this case, shouldn't it do the same thing regardless of whether you use select_related or not? i.e., shouldn't it throw that exception when you don't use select_related and try to access the same relationship from that side?

comment:10 in reply to: ↑ 9 Changed 15 months ago by aaugustin

Replying to ntucker:

If it is supposed to be throwing DoesNotExist in this case, shouldn't it do the same thing regardless of whether you use select_related or not?

Yes, the behavior must be the same with and without select_related in all cases (except for the number of queries).

I have reviewed your comments carefully and haven't found any evidence of a different behavior with and without select_related, on 1.5rc1 or master. If you see a different behavior, could you help me reproduce it:

  • either by writing a test case (examples),
  • or by pasting a definition of models and a shell session exhibiting the problem, like Russell did in the first comment?

Thanks!

comment:11 Changed 15 months ago by ntucker

Nvm, you're right. Thanks for looking at this! :)

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.