Annotated field created by subquery, referenced inside of F() generates invalid SQL

[Joey Wilhelm]

This seems like it might be related to #29214, but presented itself in a different scenario.

The following code

from django.db import models
from django.db.models import Count, F, IntegerField, OuterRef, Subquery


class Request(models.Model):
    state = models.CharField(max_length=255)


class RequestTask(models.Model):
    request = models.ForeignKey(Request, on_delete=models.CASCADE, related_name='tasks')
    state = models.CharField(max_length=255)


def find_completed_requests():
        complete = RequestTask.objects.filter(
            request=OuterRef('pk'),
            state='success'
        ).order_by().values('request')
        complete_count = complete.annotate(c=Count('*')).values('c')

        ready_to_complete = Request.objects.annotate(
            total_tasks=Count('tasks'),
            complete_tasks=Subquery(complete_count, output_field=IntegerField())
        ).filter(
            state='in_progress',
            total_tasks=F('complete_tasks')
        )

Generates the error:

Traceback (most recent call last):
  File ".venv/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
psycopg2.ProgrammingError: syntax error at or near "SELECT"
LINE 1: ...0."state" = 'success') GROUP BY U0."request_id"), SELECT COU...

This can be resolved by swapping the LHS and RHS of the fields in the final filter, to

complete_tasks=F('total_tasks')

[Tim Graham]

Do you have a mistake in the ticket? I'm getting FieldError: Cannot resolve keyword 'tasks' into field. Choices are: id, requesttask, state.

[Joey Wilhelm]

Ah, right. I forgot about the related name. This is a heavily reduced version of my actual code. Sorry about that. Fixed!

[Carlton Gibson]

OK, this reproduces for me (with a ready_to_complete.all() in find_completed_requests, obviously(?)).

I can't quite see at this exact moment if this is the same issue as #29214 or merely related, so I'll accept and make a note there too.
(It may be that they end up as duplicates.)

[Mariusz Felisiak]

I've prepared test in our test suite.

[Mariusz Felisiak]

Test.

[Mariusz Felisiak]

​PR

[Mariusz Felisiak]

In ​dd3b470:

Fixed #29542 -- Fixed invalid SQL if a Subquery from the HAVING clause is used in the GROUP BY clause.

Thanks Tim Graham for the review.

[Tim Graham <timograham@…>]

In e595a713:

Refs #29542, #30158 -- Enabled a HAVING subquery filter test on Oracle.

Now that subquery annotations aren't included in the GROUP BY unless
explicitly grouped against, the test works on Oracle.

[Tim Graham <timograham@…>]

In 3a505c7:

Refs #27149, #29542 -- Simplified subquery parentheses wrapping logic.