Opened 13 months ago

Closed 13 months ago

Last modified 13 months ago

#34603 closed Bug (wontfix)

~Q() incorrectly interpreted as full rather than empty

Reported by: Anders Kaseorg Owned by: nobody
Component: Database layer (models, ORM) Version: 4.2
Severity: Normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Since Q() is (correctly) interpreted as matching every row, its negation ~Q() should be interpreted as matching no rows. However, Django incorrectly interprets ~Q() as matching every row like Q().

>>> from django.db.models import Q
>>> from my_app.models import Client
>>> Client.objects.count()
12
>>> Client.objects.filter(Q()).count()  # good, expected 12
12
>>> Client.objects.filter(~Q()).count()  # bad, expected 0
12

Related: #5261, which introduced this incorrect behavior and added an incorrect test for it.

Change History (5)

comment:1 by Mariusz Felisiak, 13 months ago

Resolution: invalid
Status: newclosed

Thanks for the ticket, however, IMO the current behavior is correct. An empty Q() negated or not shouldn't change the WHERE statement as it doesn't contain any filters.

comment:2 by Anders Kaseorg, 13 months ago

Resolution: invalid
Status: closednew

That isn’t how logic works.

The expected meaning of .filter(~Q(k_1=v_1, k_2=v_2, …, k_n=v_n)) is to exclude those rows r such that for all 1 ≤ i ≤ n, row r satisfies k_i=v_i. If we plug in n = 0, then every row r vacuously satisfies the condition that for all 1 ≤ i ≤ n, row r satisfies k_i=v_i, so every row should be excluded.


Here’s an example. Consider this 3-column table with 8 rows:

a b c
0 0 0
0 0 1
0 1 0
0 1 1
1 0 0
1 0 1
1 1 0
1 1 1

Then .filter(~Q(a=1, b=1, c=1)) gives 7 rows—those that don’t match 1 1 1:

a b c
0 0 0
0 0 1
0 1 0
0 1 1
1 0 0
1 0 1
1 1 0

.filter(~Q(a=1, b=1)) gives 6 rows—those that don’t match 1 1 *:

a b c
0 0 0
0 0 1
0 1 0
0 1 1
1 0 0
1 0 1

.filter(~Q(a=1)) gives 4 rows—those that don’t match 1 * *:

a b c
0 0 0
0 0 1
0 1 0
0 1 1

and we should expect .filter(~Q()) to give 0 rows—those that don’t match * * *:

a b c

Another explanation comes from the invariant that the meaning of Q(…) should be unchanged if we add a criterion that matches every row. For example, in a table with an id column of nonnegative integers, .filter(~Q(k_1=v_1, k_2=v_2)) is equivalent to .filter(~Q(k_1=v_1, k_2=v_2, id__gte=0)). So, similarly, .filter(~Q()) should be equivalent to .filter(~Q(id__gte=0)). Yet Django gives

>>> Client.objects.filter(~Q()).count()
12
>>> Client.objects.filter(~Q(id__gte=0)).count()
0

comment:3 by Mariusz Felisiak, 13 months ago

Resolution: wontfix
Status: newclosed

I understand how the current logic works (there is no need for a long examples), and I'm not sure how it justifies that ~Q() should return an empty queryset. Q() is "nothing", if you will negate "nothing" you will still get "nothing". This is rather a philosophical discourse.

I appreciate you'd like to reopen the ticket, but please follow the triaging guidelines with regards to wontfix tickets and take this to DevelopersMailingList, where you'll reach a wider audience and see what other think.

comment:4 by Anders Kaseorg, 13 months ago

There’s an important difference between including nothing and excluding nothing. We can’t just say these are the same because they both involve “nothing”.

  • Q() matches every row that satisfies no particular restrictions—i.e., it matches every row. (In Python, all([]) == True.)
  • ~Q() should match everything other than the rows that satisfy no particular restrictions–i.e., it should match no rows. (In Python, not(all([])) == False.)

I know this is a dense explanation that’s hard to digest in this abstract form. But the rules of formal logic do give us a right answer here; this is not some ambiguous question of philosophy.

in reply to:  4 comment:5 by Mariusz Felisiak, 13 months ago

Replying to Anders Kaseorg:

There’s an important difference between including nothing and excluding nothing.

TBH, I don't see any difference between including and excluding a void.

I know this is a dense explanation that’s hard to digest in this abstract form. But the rules of formal logic do give us a right answer here; this is not some ambiguous question of philosophy.

As far as I'm aware, that's not true. Rules of mathematical logic don't give us a clear answer how to negate "nothing". For example, NOT NULL is NULL, it doesn't have a logical evaluation. Again, start a discussion on the Django Forum or on the mailing list, if you don't agree.

Note: See TracTickets for help on using tickets.
Back to Top