Add explicit support for Q object annotations

[Josh Smeaton]

A common pattern that has arisen with expressions is to annotate a Q object to the query without going through the Case/When ceremony.

Model.objects.annotate(boolfield=ExpressionWrapper(Q(field__gte=4), output_field=BooleanField()))

We've tried to discourage this pattern because the Case/When structure is a lot more powerful and correct. Sometimes the power isn't required and users really just want to annotate a boolean expression. Q objects and the WhereNode it resolves to aren't really proper expressions, but they could be. Case/When uses Q objects internally, which is why it's important to test Case/When for any major changes to Q and WhereNode.

Explicit support for Q object annotations should implement an implicit BooleanField as the output_field. I believe this is only required on the WhereNode since that is what is ultimately resolved. Once this happens, the above query becomes a lot nicer.

Model.objects.annotate(boolfield=Q(field__gte=4))

For this to work the following is definitely required:

• Add an output_field=BooleanField() to WhereNode
• Add a resolve_expression() to WhereNode (unsure what this should return - requires significant testing with Case/When)

Might be required:

• Add an output_field=BooleanField() to Q

Tests:

• Adding a subquery containing a Q annotation to a parent query
• Adding a subquery containing a Case/When annotation to a parent query (if it's not already tested)
• Standard tests that most other annotations already implement
• Nullable fields used in Q annotations

[Ian Foote]

I took a quick look at this and ran into circular import problems when setting output_field=BooleanField() or importing Expression (both from django.db.models). This will need some careful thought about how WhereNode and Q interact.

[Ian Foote]

While working on #470, I've created a proof of concept implementation that allows annotating lookups without requiring a Q object. (See ​https://github.com/django/django/commit/d67c858198903839b6b5bf15e04cbf7a7072190e.)

Instead of Model.objects.annotate(boolfield=Q(field__gte=4)), we would instead write something like Model.objects.annotate(boolfield=F('field') >= 4).

This is more flexible than the original proposal here because it allows comparing expressions other than fields. For example: Model.objects.annotate(boolfield=Random() > 0.5).

If this proposed API is accepted, I'll assign this ticket to myself again and polish up my proof of concept.

[Mariusz Felisiak <felisiak.mariusz@…>]

In f42ccdd8:

Fixed #27021 -- Allowed lookup expressions in annotations, aggregations, and QuerySet.filter().

Thanks Hannes Ljungberg and Simon Charette for reviews.

Co-authored-by: Mariusz Felisiak <felisiak.mariusz@…>