Aggregate functions failing when using Postgres JSON field KeyTransform

[trik]

When annotating by a nested KeyTransform applied to a JSON field, aggregate functions fail:

JSONModel.objects.annotate(history=KeyTransform('-1', 'field'))\
    .annotate(last_state=KeyTransform('state', 'history'))\
    .filter(last_state__gte=5).count()

JSONModel.objects.annotate(history=KeyTransform('-1', 'field'))\
    .annotate(last_state=KeyTransform('state', 'history'))\
    .filter(last_state__isnull=False).aggregate(Sum('last_state'))

Traceback (most recent call last):
  File "/Users/trik/Projects/3d_party/django/tests/postgres_tests/test_json.py", line 305, in test_keytransform
    .filter(last_state__gte=5).count(),
  File "/Users/trik/Projects/3d_party/django/django/db/models/query.py", line 384, in count
    return self.query.get_count(using=self.db)
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/query.py", line 494, in get_count
    number = obj.get_aggregation(using, ['__count'])['__count']
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/query.py", line 462, in get_aggregation
    outer_query.add_subquery(inner_query, using)
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/subqueries.py", line 193, in add_subquery
    self.subquery, self.sub_params = query.get_compiler(using).as_sql(with_col_aliases=True)
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/compiler.py", line 443, in as_sql
    extra_select, order_by, group_by = self.pre_sql_setup()
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/compiler.py", line 55, in pre_sql_setup
    group_by = self.get_group_by(self.select + extra_select, order_by)
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/compiler.py", line 130, in get_group_by
    if (sql, tuple(params)) not in seen:
TypeError: unhashable type: 'list'

​Failing test case

[Tim Graham]

Similar exception as #28762 but the fix there doesn't resolve this.

[ddio]

I resolved this issue when digging into nested KeyTransform.

Root Cause

1. When using nested KeyTransform, it concatenate child keys in the wrong format, i.e. a list [child1, child2]
2. Instead, according to pgsql, it should be a string like '{child1, child2, ...}'

Solution

After updating the as_sql() transformation, the issue is resolved. Will send PR soon.

[Tim Graham]

​PR (with improvements noted)

[Tim Graham]

I closed #29139 as a duplicate but that could be confirmed if a patch is provided.

[Mariusz Felisiak <felisiak.mariusz@…>]

In d87bd29:

Fixed #30335, #29139 -- Fixed crash when ordering or aggregating over a nested JSONField key transform.

[Mariusz Felisiak <felisiak.mariusz@…>]

In e85317d7:

[2.2.x] Fixed #30335, #29139 -- Fixed crash when ordering or aggregating over a nested JSONField key transform.

Backport of d87bd29c4f8dfcdf3f4a4eb8340e6770a2416fe3 from master.