AttributeError when using a GIS filter against a subquery annotation.

[Vasileios Bouzas]

I am currently building a web app based on GeoDjango and I want to check if a number of points lie within a multipolygon. To form the multipolygon, I check which areas are selected and then aggregate them into one multipolygon with the following query:

area = Area.objects.filter(id__in = area_ids).aggregate(area=Union('geom'))['area']
print(area)

SRID=4326;MULTIPOLYGON (((5.014146 52.371687, 5.013973 52.371632, ...)))

Afterwards, I retrieved all the points from the flows queryset where they are originally contained and annotate them to another chains queryset under a new name ('producer_geom'):

subq = flows.filter(chains_id=OuterRef('pk'),
                    origin_role='producer')

chains = chains.annotate(pro_geom=
                         Subquery(subq.values('origin__administrative_location__geom'))
                         )

The result is the following:

print(chains.values_list('id', 'pro_geom'))
<QuerySet [(10804, <Point object at 0x7f656e633de0>), (10803, <Point object at 0x7f656e649ab0>), (10802, <Point object at 0x7f656e649bc0>), '...(remaining elements truncated)...']>

But when I try to filter the chains aftewards, to collect the ids of the chains that the producer point lies with the area multipolygon,

chains = chains.filter(pro_geom__within=area)

I get the following error message:

Traceback (most recent call last):
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/home/geofluxus/.local/lib/python3.6/site-packages/rest_framework/viewsets.py", line 116, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/geofluxus/.local/lib/python3.6/site-packages/reversion/views.py", line 35, in do_revision_view
    response = func(request, *args, **kwargs)
  File "/home/geofluxus/.local/lib/python3.6/site-packages/rest_framework/views.py", line 495, in dispatch
    response = self.handle_exception(exc)
  File "/home/geofluxus/.local/lib/python3.6/site-packages/rest_framework/views.py", line 455, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/home/geofluxus/.local/lib/python3.6/site-packages/rest_framework/views.py", line 466, in raise_uncaught_exception
    raise exc
  File "/home/geofluxus/.local/lib/python3.6/site-packages/rest_framework/views.py", line 492, in dispatch
    response = handler(request, *args, **kwargs)
  File "/home/geofluxus/Desktop/geoFluxus/repair/apps/utils/views.py", line 36, in create
    return self.post_get(request, **kwargs)
  File "/home/geofluxus/Desktop/geoFluxus/repair/apps/asmfa/views/flowfilter.py", line 206, in post_get
    queryset = self.filter_chain(queryset, filter_chains, keyflow)
  File "/home/geofluxus/Desktop/geoFluxus/repair/apps/asmfa/views/flowfilter.py", line 356, in filter_chain
    queryset = build_chain_filter(sub_filter, queryset, keyflow)
  File "/home/geofluxus/Desktop/geoFluxus/repair/apps/asmfa/views/flowfilter.py", line 151, in build_chain_filter
    chains = list(chains.values_list('id', flat=True))
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/db/models/query.py", line 274, in __iter__
    self._fetch_all()
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/db/models/query.py", line 1242, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/db/models/query.py", line 182, in __iter__
    for row in compiler.results_iter(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size):
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1052, in results_iter
    results = self.execute_sql(MULTI, chunked_fetch=chunked_fetch, chunk_size=chunk_size)
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1087, in execute_sql
    sql, params = self.as_sql()
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 489, in as_sql
    where, w_params = self.compile(self.where) if self.where is not None else ("", [])
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 405, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/db/models/sql/where.py", line 81, in as_sql
    sql, params = compiler.compile(child)
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 405, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/db/models/sql/where.py", line 81, in as_sql
    sql, params = compiler.compile(child)
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 405, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/db/models/sql/where.py", line 81, in as_sql
    sql, params = compiler.compile(child)
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 405, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/home/geofluxus/.local/lib/python3.6/site-packages/django/contrib/gis/db/models/lookups.py", line 79, in as_sql
    sql_params.extend(rhs_params)
AttributeError: 'tuple' object has no attribute 'extend'

Reproduced with Django v2.2.6 & v2.2.7

[Simon Charette]

Likely caused by our badly defined Expression/Lookup.as_sql signature where we sometimes return params as a list and sometimes as a tuple.

Given Subquery.as_sql returns them as a tuple, through Query.as_sql, we likely want to adjust GISLookup.as_sql here.

Could you give the following patch a try Vasileios

django/contrib/gis/db/models/lookups.py

@@ -75 +75 @@
         return connection.ops.gis_operators[self.lookup_name]
 
     def as_sql(self, compiler, connection):
-        lhs_sql, sql_params = self.process_lhs(compiler, connection)
+        lhs_sql, lhs_params = self.process_lhs(compiler, connection)
         rhs_sql, rhs_params = self.process_rhs(compiler, connection)
-        sql_params.extend(rhs_params)
+        sql_params = tuple(lhs_params) + tuple(rhs_params)
 
         template_params = {'lhs': lhs_sql, 'rhs': rhs_sql, 'value': '%s', **self.template_params}
         rhs_op = self.get_rhs_op(connection, rhs_sql)

[Mariusz Felisiak <felisiak.mariusz@…>]

In 0290e01:

Fixed #31002 -- Fixed GIS lookups crash against a subquery annotation.

Thanks Vasileios Bouzas for the report.