Opened 12 months ago

Last modified 6 weeks ago

#29139 assigned Bug

Aggregate functions failing when using Postgres JSON field KeyTransform

Reported by: trik Owned by: ddio
Component: contrib.postgres Version: 2.0
Severity: Normal Keywords: json keytransform unhashable
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no


When annotating by a nested KeyTransform applied to a JSON field, aggregate functions fail:

JSONModel.objects.annotate(history=KeyTransform('-1', 'field'))\
    .annotate(last_state=KeyTransform('state', 'history'))\

JSONModel.objects.annotate(history=KeyTransform('-1', 'field'))\
    .annotate(last_state=KeyTransform('state', 'history'))\
Traceback (most recent call last):
  File "/Users/trik/Projects/3d_party/django/tests/postgres_tests/", line 305, in test_keytransform
  File "/Users/trik/Projects/3d_party/django/django/db/models/", line 384, in count
    return self.query.get_count(using=self.db)
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/", line 494, in get_count
    number = obj.get_aggregation(using, ['__count'])['__count']
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/", line 462, in get_aggregation
    outer_query.add_subquery(inner_query, using)
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/", line 193, in add_subquery
    self.subquery, self.sub_params = query.get_compiler(using).as_sql(with_col_aliases=True)
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/", line 443, in as_sql
    extra_select, order_by, group_by = self.pre_sql_setup()
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/", line 55, in pre_sql_setup
    group_by = self.get_group_by( + extra_select, order_by)
  File "/Users/trik/Projects/3d_party/django/django/db/models/sql/", line 130, in get_group_by
    if (sql, tuple(params)) not in seen:
TypeError: unhashable type: 'list'

Failing test case

Change History (6)

comment:1 Changed 12 months ago by Tim Graham

Triage Stage: UnreviewedAccepted

Similar exception as #28762 but the fix there doesn't resolve this.

comment:2 Changed 12 months ago by Piotr Domański

Owner: set to Piotr Domański
Status: newassigned

comment:3 Changed 10 months ago by Piotr Domański

Owner: Piotr Domański deleted
Status: assignednew

comment:4 Changed 7 months ago by ddio

Owner: set to ddio
Status: newassigned

I resolve this issue when digging into nested KeyTransform.

Root Cause

  1. When using nested KeyTransform, it concatenate child keys in the wrong format, i.e. a list [child1, child2]
  2. Instead, according to pgsql, it should be a string like '{child1, child2, ...}'


After updating the as_sql() transformation, the issue is resolved. Will send PR soon.

Version 0, edited 7 months ago by ddio (next)

comment:5 Changed 6 months ago by Tim Graham

Has patch: set
Patch needs improvement: set

PR (with improvements noted)

comment:6 Changed 6 weeks ago by Tim Graham

I closed #30085 as a duplicate but that could be confirmed if a patch is provided.

Last edited 6 weeks ago by Tim Graham (previous) (diff)
Note: See TracTickets for help on using tickets.
Back to Top