Opened 3 weeks ago

Closed 12 days ago

#37438 closed Bug (fixed)

Negated XOR conditions incorrectly use INNER JOIN for nullable relations

Reported by: Pravin Owned by: Pravin
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords: XOR, JoinPromoter
Cc: Pravin Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Given the following models:

class Detail(models.Model):
    first = models.CharField(max_length=20)


class Item(models.Model):
    detail = models.ForeignKey(
        Detail,
        models.CASCADE,
        null=True,
    )

The following query should return both items:

from django.db.models import Q

detail = Detail.objects.create(first="match")
with_detail = Item.objects.create(detail=detail)
without_detail = Item.objects.create()

items = Item.objects.filter(
    ~(
        Q(detail__first__isnull=True)
        ^ Q(pk=without_detail.pk)
    )
).order_by("pk")

print(list(items.values_list("pk", flat=True)))

For with_detail, detail__first__isnull and
pk=without_detail.pk are both false.

For without_detail, the missing detail produces a NULL
value and its primary key matches without_detail.pk, so both conditions
are true.

A negated XOR matches when both conditions have the same result. Therefore,
both items should be returned.

Expected result:

[with_detail.pk, without_detail.pk]

Actual result:

[with_detail.pk]

JoinPromoter doesn't handle XOR separately. For join promotion,
a negated XOR is treated as an AND, so an INNER JOIN is considered
safe. The inner join removes without_detail before the conditions are
evaluated.

Change History (7)

comment:1 by Simon Charette, 3 weeks ago

Triage Stage: Unreviewed → Accepted

​I confirmed this has been a bug since the introduction of XOR in c6b4d62fa2c7f73b87f6ae7e8cf1d64ee5312dc5.

The bug effectively lies in JoinPromoter which was never made aware or it.

comment:2 by Pravin, 3 weeks ago

Has patch: set

comment:3 by Simon Charette, 3 weeks ago

Patch needs improvement: set

Left some comments regarding a potential better host for tests and a suggestion to adjust the effective_connector flow.

comment:4 by Pravin, 3 weeks ago

Patch needs improvement: unset

Thanks, I moved the tests and update the effective_connector flow.

comment:5 by Simon Charette, 2 weeks ago

Triage Stage: Accepted → Ready for checkin

Thank you for the adjustments.

comment:6 by Pravin, 2 weeks ago

Receiving approval from you is more rewarding itself. Thank you for review.

comment:7 by Sarah Boyce <42296566+sarahboyce@…>, 12 days ago

Resolution: → fixed
Status: assigned → closed

In 77eea99:

Fixed #37438 -- Fixed join promotion for XOR conditions.

JoinPromoter didn't handle XOR conditions. Because of this, a negated
XOR could use an INNER JOIN and remove rows that should be returned.

Thanks Simon Charette and Sarah Boyce for review.

Note: See TracTickets for help on using tickets.
Back to Top