Opened 6 years ago

Closed 5 years ago

Last modified 5 years ago

#14648 closed Bug (fixed)

Annotated date querysets fail if spatial backend is used

Reported by: codysoyland Owned by: jbronn
Component: GIS Version: 1.2
Severity: Normal Keywords:
Cc: me@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: yes
Easy pickings: UI/UX:

Description

Annotated DateQuerysets fail if django.contrib.gis is installed. By that I mean this queryset will throw an exception:

Bug.objects.annotate(num_patches=Count('patches')).dates('created', 'year')

Note that I'm not intentionally running dates() on an annotated queryset (that would normally be pointless); this is happening inside the date_based.archive_index generic view and I need the aggregate annotation in the main queryset.

My example project at https://github.com/codysoyland/django-gis-bug-example/ demonstrates this.
See the tests: https://github.com/codysoyland/django-gis-bug-example/blob/master/bughouse/tests.py

The test output is:

======================================================================
ERROR: test_date_queryset (bughouse.tests.DateQuerysetTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/csoyland/projects/buggy/bughouse/tests.py", line 19, in test_date_queryset
    len(Bug.objects.annotate(num_patches=Count('patches')).dates('created', 'year')),
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/db/models/query.py", line 81, in __len__
    self._result_cache = list(self.iterator())
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 948, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 717, in execute_sql
    sql, params = self.as_sql()
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 56, in as_sql
    out_cols = self.get_columns(with_col_aliases)
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/contrib/gis/db/models/sql/compiler.py", line 78, in get_columns
    for alias, aggregate in self.query.aggregate_select.items()
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/contrib/gis/db/models/sql/compiler.py", line 205, in get_extra_select_format
    if alias in self.query.custom_select:
AttributeError: 'DateQuery' object has no attribute 'custom_select'

Adding the GeoManager to this model results in a different error:

======================================================================
ERROR: test_date_queryset (bughouse.tests.DateQuerysetTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/csoyland/projects/buggy/bughouse/tests.py", line 19, in test_date_queryset
    len(Bug.objects.annotate(num_patches=Count('patches')).dates('created', 'year')),
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/db/models/query.py", line 81, in __len__
    self._result_cache = list(self.iterator())
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 952, in results_iter
    date = self.resolve_columns(row, fields)[offset]
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/contrib/gis/db/models/sql/compiler.py", line 192, in resolve_columns
    for v, a in izip(row[rn_offset:index_start], aliases)]
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/db/models/sql/query.py", line 305, in convert_values
    return connection.ops.convert_values(value, field)
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/db/backends/__init__.py", line 441, in convert_values
    internal_type = field.get_internal_type()
AttributeError: 'NoneType' object has no attribute 'get_internal_type'

Attachments (2)

14648.1.diff (1.6 KB) - added by jbronn 6 years ago.
14648.2.diff (6.3 KB) - added by jbronn 5 years ago.

Download all attachments as: .zip

Change History (11)

Changed 6 years ago by jbronn

Attachment: 14648.1.diff added

comment:1 Changed 6 years ago by jbronn

Has patch: set
Needs documentation: unset
Needs tests: set
Owner: changed from nobody to jbronn
Patch needs improvement: set
Status: newassigned

The crux of the problem is that the methods of GeoSQLCompiler are being called on normal Query objects, rather than with GeoQuery as it was designed. Thus, attributes that it expects to be available (custom_select, extra_select_fields), are not.

Attached is an initial patch, tests and more investigation are necessary, especially for Oracle -- as it is a backend with its own SQLCompiler class that implements resolve_columns.

comment:2 Changed 6 years ago by cmarrero

milestone: 1.3

comment:3 Changed 6 years ago by anonymous

Cc: me@… added

comment:4 Changed 6 years ago by Valentin Golev

A little investigation shows what this was broken after changesets 14716/14717

comment:5 Changed 6 years ago by Russell Keith-Magee

Triage Stage: UnreviewedAccepted

comment:6 Changed 5 years ago by James Addison

Severity: Normal
Type: Bug

comment:7 Changed 5 years ago by James Addison

milestone: 1.3

Changed 5 years ago by jbronn

Attachment: 14648.2.diff added

comment:8 Changed 5 years ago by jbronn

Resolution: fixed
Status: assignedclosed

In [16796]:

Fixed #14648 -- Fixed annotated date querysets when GeoManager is used. Thanks, codysoyland, for the bug report.

comment:9 Changed 5 years ago by jbronn

In [16844]:

[1.3.X] Fixed #14648 -- Fixed annotated date querysets when GeoManager is used. Thanks, codysoyland, for the bug report.

Backport of r16796 from trunk.

Note: See TracTickets for help on using tickets.
Back to Top