Opened 5 years ago

Closed 4 years ago

Last modified 4 years ago

#16408 closed Bug (fixed)

Query result types not being converted with Spatialite

Reported by: Antonio Eugenio Burriel <aeburriel@…> Owned by: jbronn
Component: GIS Version: 1.5-beta-1
Severity: Normal Keywords: regression
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

While making a model gis-aware, I've noticed the following wrong behavior with the Spatialite backend (works as expected with Postgis).

Consider the following model:

from django.contrib.gis.db import models
class Demo(models.Model):
  timestamp = models.DateTimeField(auto_now_add=True)
  number = models.DecimalField(decimal_places=2, max_digits=5)
  # objects = models.GeoManager()	# Disabled on purpose

Open a django shell and populate the model:

from demo.models import Demo
from django.contrib.gis.db.models import *

Demo(number="13.22").save()

A simple query

>>> Demo.objects.all().values()
[{'timestamp': datetime.datetime(2011, 7, 4, 16, 8, 26, 297486), 'id': 1, 'number': Decimal('13.22')}]

So far, so good

>>> Demo.objects.aggregate(Min("number"), Min("timestamp"))
{'number__min': Decimal('13.22'), 'timestamp__min': datetime.datetime(2011, 7, 4, 16, 8, 26, 297486)}

Just as expected.

Let's do it again, this time enabling the GeoManager:

from django.contrib.gis.db import models
class Demo(models.Model):
  timestamp = models.DateTimeField(auto_now_add=True)
  number = models.DecimalField(decimal_places=2, max_digits=5)
  objects = models.GeoManager()
from demo.models import Demo
from django.contrib.gis.db.models import *
>>> Demo.objects.all().values()
[{'timestamp': datetime.datetime(2011, 7, 4, 16, 8, 26, 297486), 'id': 1, 'number': Decimal('13.22')}]

The same...

Now...

>>> Demo.objects.aggregate(Min("number"), Min("timestamp"))
{'number__min': 13.220000000000001, 'timestamp__min': u'2011-07-04 16:08:26.297486'}

Whoops! float and string instead of Decimal and datetime!
See the difference? With GeoManager. convert_values() seems not to be called, at least for DateQuerySets and aggregates.

Attachments (4)

16408.1.diff (5.5 KB) - added by jbronn 5 years ago.
16408.2.diff (5.5 KB) - added by jbronn 5 years ago.
16408.3.diff (10.0 KB) - added by jbronn 5 years ago.
16408-4.diff (1.7 KB) - added by Claude Paroz 4 years ago.
Call parent convert_values

Download all attachments as: .zip

Change History (14)

comment:1 Changed 5 years ago by Aymeric Augustin

Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset
Triage Stage: UnreviewedAccepted

I didn't compile spatialite to confirm the bug, but the report is sufficiently detailed to accept the ticket.

This doesn't look related to #15040 and #15169.

comment:2 Changed 5 years ago by jbronn

Has patch: set
milestone: 1.4
Owner: changed from nobody to jbronn
Status: newassigned

Changed 5 years ago by jbronn

Attachment: 16408.1.diff added

Changed 5 years ago by jbronn

Attachment: 16408.2.diff added

Changed 5 years ago by jbronn

Attachment: 16408.3.diff added

comment:3 Changed 5 years ago by jbronn

Resolution: fixed
Status: assignedclosed

In [16749]:

Fixed #16408 -- Fixed conversion of dates, and other problems with the SpatiaLite backend.

comment:4 Changed 5 years ago by jbronn

In [16751]:

[1.3.X] Fixed #16408 -- Fixed conversion of dates, and other problems with the SpatiaLite backend.

Backport of r16749 and r16750 from trunk.

comment:5 Changed 5 years ago by Jacob

milestone: 1.4

Milestone 1.4 deleted

comment:6 in reply to:  5 Changed 4 years ago by Mathijs Dumon

Keywords: regression added
Resolution: fixed
Status: closedreopened
Version: 1.31.5-beta-1

This issue has re-appeared on django 1.5 b2.

I checked the source files and it appears the spatialite backend is not patched for this issue in this version, and is still using compilers etc. from django.contrib.gis.db.models.sql.compile instead of django.contrib.gis.db.models.spatialite.compile (which is missing)
merging the previous commits would solve it I guess...

I stand corrected, this seems to have moved down, outside the spatialite backend into the base sql backend, my appologies. I can correctly get the values as datatime objects in a shell.
The bug occurs when using the 'date_hierarchy' on an admin model, checking revealed the aggregate function still does not seem to call convert_values()

Last edited 4 years ago by Mathijs Dumon (previous) (diff)

comment:7 Changed 4 years ago by Mathijs Dumon

I should probably add the error occurs here:
/usr/local/lib/python2.7/dist-packages/django/contrib/admin/templatetags/admin_list.py in date_hierarchy, line 310

And that the problem disappears if I remove the objects = models.GeoManager() line from my model.

Changed 4 years ago by Claude Paroz

Attachment: 16408-4.diff added

Call parent convert_values

comment:8 Changed 4 years ago by Claude Paroz

Could you please test the attached patch?

comment:9 Changed 4 years ago by Claude Paroz <claude@…>

Resolution: fixed
Status: reopenedclosed

In 0907d3c6f511ef3dc68d381389a32b9b54ae4be7:

Fixed #16408 -- Re-fixed value conversion with Spatialite backend

comment:10 Changed 4 years ago by Claude Paroz <claude@…>

In 4f67ab63761a949a4ee8dab5ae38f23b32dc56e5:

[1.5.x] Fixed #16408 -- Re-fixed value conversion with Spatialite backend

Backport of 0907d3c6f from master.

Note: See TracTickets for help on using tickets.
Back to Top