Opened 6 months ago

Last modified 3 weeks ago

#29139 assigned Bug

Aggregate functions failing when using Postgres JSON field KeyTransform

Reported by: trik Owned by: ddio
Component: contrib.postgres Version: 2.0
Severity: Normal Keywords: json keytransform unhashable
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

When annotating by a nested KeyTransform applied to a JSON field, aggregate functions fail:

JSONModel.objects.annotate(history=KeyTransform('-1', 'field'))\
    .annotate(last_state=KeyTransform('state', 'history'))\
    .filter(last_state__gte=5).count()

JSONModel.objects.annotate(history=KeyTransform('-1', 'field'))\
    .annotate(last_state=KeyTransform('state', 'history'))\
    .filter(last_state__isnull=False).aggregate(Sum('last_state'))
Traceback (most recent call last):
  File "/Users/trik/Projects/3d_party/django/tests/postgres_tests/test_json.py", line 305, in test_keytransform
    .filter(last_state__gte=5).count(),
  File "/Users/trik/Projects/3d_party/django/django/db/models/query.py", line 384, in count
    return self.query.get_count(using=self.db)
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/query.py", line 494, in get_count
    number = obj.get_aggregation(using, ['__count'])['__count']
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/query.py", line 462, in get_aggregation
    outer_query.add_subquery(inner_query, using)
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/subqueries.py", line 193, in add_subquery
    self.subquery, self.sub_params = query.get_compiler(using).as_sql(with_col_aliases=True)
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/compiler.py", line 443, in as_sql
    extra_select, order_by, group_by = self.pre_sql_setup()
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/compiler.py", line 55, in pre_sql_setup
    group_by = self.get_group_by(self.select + extra_select, order_by)
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/compiler.py", line 130, in get_group_by
    if (sql, tuple(params)) not in seen:
TypeError: unhashable type: 'list'

Failing test case

Change History (4)

comment:1 Changed 6 months ago by Tim Graham

Triage Stage: UnreviewedAccepted

Similar exception as #28762 but the fix there doesn't resolve this.

comment:2 Changed 6 months ago by Piotr Domański

Owner: set to Piotr Domański
Status: newassigned

comment:3 Changed 4 months ago by Piotr Domański

Owner: Piotr Domański deleted
Status: assignednew

comment:4 Changed 3 weeks ago by ddio

Owner: set to ddio
Status: newassigned

I resolved this issue when digging into nested KeyTransform.

Root Cause

  1. When using nested KeyTransform, it concatenate child keys in the wrong format, i.e. a list [child1, child2]
  2. Instead, according to pgsql, it should be a string like '{child1, child2, ...}'

Solution

After updating the as_sql() transformation, the issue is resolved. Will send PR soon.

Last edited 3 weeks ago by ddio (previous) (diff)
Note: See TracTickets for help on using tickets.
Back to Top