Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#27014 closed Bug (fixed)

Raster support for spatial lookup breaks filtering by annotations

Reported by: Tristen Georgiou Owned by: nobody
Component: GIS Version: 1.10
Severity: Release blocker Keywords: raster, spatial
Cc: Daniel Wiesmann Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

This maybe a very specific edge case; I noticed that my website unit tests break when I upgraded to 1.10 and it appears to occur when I filter spatially and then annotate and then filter upon that annotation. (using Python 3, and the django.contrib.gis.db.backends.postgis backend)

Here's code to reproduce:
models.py

from django.contrib.gis.db import models

class Event(models.Model):
    enabled = models.BooleanField(default=True, db_index=True)
    location = models.PointField(geography=True)
    distance = models.IntegerField(default=5)

    objects = models.GeoManager()

And a unit test to show the error:
tests.py

from django.contrib.gis.measure import D
from django.db.models import F
from django.test import TestCase
from django.contrib.gis.geos import Point
from django.contrib.gis.db.models.functions import Distance

from polls.models import Event

class SampleTest(TestCase):
    def test_something(self):
        centre = Point(0.0, 0.0)
        query = Event.objects.filter(location__dwithin=(centre, D(km=5)))
        query = query.annotate(
            actual_distance=Distance('location', centre)
        ).filter(
            actual_distance__lte=F('distance') * 1000)
        print(list(query.all()))

And stacktrace:

Traceback (most recent call last):
  File "/Users/tristeng/PycharmProjects/rastertest/polls/tests.py", line 18, in test_something
    print(list(query.all()))
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/db/models/query.py", line 256, in __iter__
    self._fetch_all()
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/db/models/query.py", line 1085, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/db/models/query.py", line 54, in __iter__
    results = compiler.execute_sql()
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 824, in execute_sql
    sql, params = self.as_sql()
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 376, in as_sql
    where, w_params = self.compile(self.where) if self.where is not None else ("", [])
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 353, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/db/models/sql/where.py", line 79, in as_sql
    sql, params = compiler.compile(child)
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 353, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/db/models/lookups.py", line 155, in as_sql
    lhs_sql, params = self.process_lhs(compiler, connection)
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/db/models/lookups.py", line 146, in process_lhs
    compiler, connection, lhs)
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/db/models/lookups.py", line 67, in process_lhs
    return compiler.compile(lhs)
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 351, in compile
    sql, params = vendor_impl(self, self.connection)
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/contrib/gis/db/models/functions.py", line 252, in as_postgresql
    return super(Distance, self).as_sql(compiler, connection)
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/contrib/gis/db/models/functions.py", line 43, in as_sql
    return super(GeoFunc, self).as_sql(compiler, connection)
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/db/models/expressions.py", line 521, in as_sql
    arg_sql, arg_params = compiler.compile(arg)
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 351, in compile
    sql, params = vendor_impl(self, self.connection)
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/contrib/gis/db/models/functions.py", line 82, in as_postgresql
    self.value = connection.ops.Adapter(self.value, geography=self.geography)
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/adapter.py", line 26, in __init__
    self.ewkb = to_pgraster(obj)
  File "/Users/tristeng/venvs/djangotest/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/pgraster.py", line 123, in to_pgraster
    1, 0, len(rast.bands), rast.scale.x, rast.scale.y,
AttributeError: 'PostGISAdapter' object has no attribute 'bands'

It looks like PostGISAdapter initializer needs to handle the additional case where the type of 'obj' passed into the constructor is actually of it's own type, otherwise it assumes its a PGRaster and attempts to convert it; do we need to check isinstance(obj, PostGISAdapter) and handle accordingly?

class PostGISAdapter(object):
    def __init__(self, obj, geography=False):
        """
        Initialize on the spatial object.
        """
        self.is_geometry = isinstance(obj, Geometry)

        # Getting the WKB (in string form, to allow easy pickling of
        # the adaptor) and the SRID from the geometry or raster.
        if self.is_geometry:
            self.ewkb = bytes(obj.ewkb)
            self._adapter = Binary(self.ewkb)
        else:
            self.ewkb = to_pgraster(obj)

        self.srid = obj.srid
        self.geography = geography

Change History (7)

comment:1 Changed 3 years ago by Claude Paroz

Severity: NormalRelease blocker
Triage Stage: UnreviewedAccepted

comment:2 Changed 3 years ago by Tim Graham

Cc: Daniel Wiesmann added

Daniel, could you take a look? I believe bbfad84dd980a97174c3b061a3d1b5f1373c380d is the relevant commit.

comment:3 Changed 3 years ago by Sean Mc Allister

I just ran into this problem and have created a minimal test case to reproduce it here:
https://github.com/mcallistersean/django/commit/6a2e6c69feb27b2d097020959151a12dfe65963b

comment:4 Changed 3 years ago by Daniel Wiesmann

I am on holidays until the end of this week, but I'll have a look at the problem as soon as I am back early next week.

comment:5 Changed 3 years ago by Tim Graham

Has patch: set
Triage Stage: AcceptedReady for checkin

comment:6 Changed 3 years ago by Tim Graham <timograham@…>

Resolution: fixed
Status: newclosed

In 89f17e7c:

Fixed #27014 -- Fixed annotations with database functions on PostGIS.

Thanks Sean Mc Allister for providing a test.

comment:7 Changed 3 years ago by Tim Graham <timograham@…>

In 20119813:

[1.10.x] Fixed #27014 -- Fixed annotations with database functions on PostGIS.

Thanks Sean Mc Allister for providing a test.

Backport of 89f17e7caf0d1e70417253d135607c9fd6ebac56 from master

Note: See TracTickets for help on using tickets.
Back to Top