Opened 2 years ago

Last modified 2 months ago

#26056 new New feature

ArrayField does not work with ValueListQuerySet

Reported by: Przemek Owned by:
Component: contrib.postgres Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description (last modified by Tim Graham)

Basically queries of type:

A.objects.filter(array_field__overlap=B.objects.filter(foo).values_list('id', flat=True))

fail at Python level:

Traceback (most recent call last):
  File "failing.py", line 9, in <module>
    UserList.objects.filter(users__overlap=User.objects.all().values_list('id', flat=True)).count()
  File "/home/przemek/.virtualenvs/italamo/lib/python2.7/site-packages/Django-1.8.5-py2.7.egg/django/db/models/query.py", line 318, in count
    return self.query.get_count(using=self.db)
  File "/home/przemek/.virtualenvs/italamo/lib/python2.7/site-packages/Django-1.8.5-py2.7.egg/django/db/models/sql/query.py", line 466, in get_count
    number = obj.get_aggregation(using, ['__count'])['__count']
  File "/home/przemek/.virtualenvs/italamo/lib/python2.7/site-packages/Django-1.8.5-py2.7.egg/django/db/models/sql/query.py", line 447, in get_aggregation
    result = compiler.execute_sql(SINGLE)
  File "/home/przemek/.virtualenvs/italamo/lib/python2.7/site-packages/Django-1.8.5-py2.7.egg/django/db/models/sql/compiler.py", line 829, in execute_sql
    sql, params = self.as_sql()
  File "/home/przemek/.virtualenvs/italamo/lib/python2.7/site-packages/Django-1.8.5-py2.7.egg/django/db/models/sql/compiler.py", line 387, in as_sql
    where, w_params = self.compile(self.query.where)
  File "/home/przemek/.virtualenvs/italamo/lib/python2.7/site-packages/Django-1.8.5-py2.7.egg/django/db/models/sql/compiler.py", line 357, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/home/przemek/.virtualenvs/italamo/lib/python2.7/site-packages/Django-1.8.5-py2.7.egg/django/db/models/sql/where.py", line 104, in as_sql
    sql, params = compiler.compile(child)
  File "/home/przemek/.virtualenvs/italamo/lib/python2.7/site-packages/Django-1.8.5-py2.7.egg/django/db/models/sql/compiler.py", line 357, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/home/przemek/.virtualenvs/italamo/lib/python2.7/site-packages/Django-1.8.5-py2.7.egg/django/contrib/postgres/fields/array.py", line 183, in as_sql
    sql, params = super(ArrayOverlap, self).as_sql(qn, connection)
  File "/home/przemek/.virtualenvs/italamo/lib/python2.7/site-packages/Django-1.8.5-py2.7.egg/django/contrib/postgres/lookups.py", line 8, in as_sql
    params = lhs_params + rhs_params
TypeError: can only concatenate list (not "tuple") to list

Toy project to reproduce this behavior can be found here: https://github.com/CGenie/django_array_join_fail

This fails in 1.8 as well as in 1.9.

Change History (7)

comment:1 Changed 2 years ago by Simon Charette

Triage Stage: UnreviewedAccepted
Type: BugNew feature
Version: 1.9master

Accepting as a new feature by assuming you meant B.objects.filter(foo).values('pk') in your reported example.

The implementation could simply use the PostgreSQL array() function to wrap the queryset.

comment:2 Changed 2 years ago by Tim Graham

Description: modified (diff)

comment:3 Changed 11 months ago by Mads Jensen

Has patch: set
Needs documentation: set

PR it follows the suggestion to wrap a query inside array. Needs documentation is marked, since the release notes need to be updated, and probably also a note in the documentation itself.

comment:4 Changed 11 months ago by Simon Charette

Patch needs improvement: set

comment:5 Changed 10 months ago by Mads Jensen

Needs documentation: unset

It has been reworked but I suppose other lookups in django.contrib.postgres will want a similar functionality.

comment:6 Changed 10 months ago by Alex3917

Does the new Subquery functionality in Django 1.11 allow for a similar result? E.g. would this work?:

qs_b = B.objects.filter(foo)
A.objects.filter(array_field__overlap=Subquery(qs_b.values_list('id', flat=True)))

EDIT: This approach does not work.

Last edited 8 months ago by Alex3917 (previous) (diff)

comment:7 Changed 2 months ago by kosz85

I posted solution here: https://github.com/django/django/pull/7838#issuecomment-337223376
It's Array with Subquery, just pure subquery isn't enough.

Note: See TracTickets for help on using tickets.
Back to Top