Code

Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#14648 closed Bug (fixed)

Annotated date querysets fail if spatial backend is used

Reported by: codysoyland Owned by: jbronn
Component: GIS Version: 1.2
Severity: Normal Keywords:
Cc: me@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: yes
Easy pickings: UI/UX:

Description

Annotated DateQuerysets fail if django.contrib.gis is installed. By that I mean this queryset will throw an exception:

Bug.objects.annotate(num_patches=Count('patches')).dates('created', 'year')

Note that I'm not intentionally running dates() on an annotated queryset (that would normally be pointless); this is happening inside the date_based.archive_index generic view and I need the aggregate annotation in the main queryset.

My example project at https://github.com/codysoyland/django-gis-bug-example/ demonstrates this.
See the tests: https://github.com/codysoyland/django-gis-bug-example/blob/master/bughouse/tests.py

The test output is:

======================================================================
ERROR: test_date_queryset (bughouse.tests.DateQuerysetTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/csoyland/projects/buggy/bughouse/tests.py", line 19, in test_date_queryset
    len(Bug.objects.annotate(num_patches=Count('patches')).dates('created', 'year')),
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/db/models/query.py", line 81, in __len__
    self._result_cache = list(self.iterator())
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 948, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 717, in execute_sql
    sql, params = self.as_sql()
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 56, in as_sql
    out_cols = self.get_columns(with_col_aliases)
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/contrib/gis/db/models/sql/compiler.py", line 78, in get_columns
    for alias, aggregate in self.query.aggregate_select.items()
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/contrib/gis/db/models/sql/compiler.py", line 205, in get_extra_select_format
    if alias in self.query.custom_select:
AttributeError: 'DateQuery' object has no attribute 'custom_select'

Adding the GeoManager to this model results in a different error:

======================================================================
ERROR: test_date_queryset (bughouse.tests.DateQuerysetTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/csoyland/projects/buggy/bughouse/tests.py", line 19, in test_date_queryset
    len(Bug.objects.annotate(num_patches=Count('patches')).dates('created', 'year')),
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/db/models/query.py", line 81, in __len__
    self._result_cache = list(self.iterator())
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 952, in results_iter
    date = self.resolve_columns(row, fields)[offset]
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/contrib/gis/db/models/sql/compiler.py", line 192, in resolve_columns
    for v, a in izip(row[rn_offset:index_start], aliases)]
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/db/models/sql/query.py", line 305, in convert_values
    return connection.ops.convert_values(value, field)
  File "/Users/csoyland/.virtualenvs/generic/lib/python2.6/site-packages/django/db/backends/__init__.py", line 441, in convert_values
    internal_type = field.get_internal_type()
AttributeError: 'NoneType' object has no attribute 'get_internal_type'

Attachments (2)

14648.1.diff (1.6 KB) - added by jbronn 3 years ago.
14648.2.diff (6.3 KB) - added by jbronn 3 years ago.

Download all attachments as: .zip

Change History (11)

Changed 3 years ago by jbronn

comment:1 Changed 3 years ago by jbronn

  • Has patch set
  • Needs documentation unset
  • Needs tests set
  • Owner changed from nobody to jbronn
  • Patch needs improvement set
  • Status changed from new to assigned

The crux of the problem is that the methods of GeoSQLCompiler are being called on normal Query objects, rather than with GeoQuery as it was designed. Thus, attributes that it expects to be available (custom_select, extra_select_fields), are not.

Attached is an initial patch, tests and more investigation are necessary, especially for Oracle -- as it is a backend with its own SQLCompiler class that implements resolve_columns.

comment:2 Changed 3 years ago by cmarrero

  • milestone set to 1.3

comment:3 Changed 3 years ago by anonymous

  • Cc me@… added

comment:4 Changed 3 years ago by valyagolev

A little investigation shows what this was broken after changesets 14716/14717

comment:5 Changed 3 years ago by russellm

  • Triage Stage changed from Unreviewed to Accepted

comment:6 Changed 3 years ago by jaddison

  • Severity set to Normal
  • Type set to Bug

comment:7 Changed 3 years ago by jaddison

  • milestone 1.3 deleted

Changed 3 years ago by jbronn

comment:8 Changed 3 years ago by jbronn

  • Resolution set to fixed
  • Status changed from assigned to closed

In [16796]:

Fixed #14648 -- Fixed annotated date querysets when GeoManager is used. Thanks, codysoyland, for the bug report.

comment:9 Changed 3 years ago by jbronn

In [16844]:

[1.3.X] Fixed #14648 -- Fixed annotated date querysets when GeoManager is used. Thanks, codysoyland, for the bug report.

Backport of r16796 from trunk.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.