Code

Opened 3 years ago

Closed 16 months ago

Last modified 16 months ago

#16408 closed Bug (fixed)

Query result types not being converted with Spatialite

Reported by: Antonio Eugenio Burriel <aeburriel@…> Owned by: jbronn
Component: GIS Version: 1.5-beta-1
Severity: Normal Keywords: regression
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

While making a model gis-aware, I've noticed the following wrong behavior with the Spatialite backend (works as expected with Postgis).

Consider the following model:

from django.contrib.gis.db import models
class Demo(models.Model):
  timestamp = models.DateTimeField(auto_now_add=True)
  number = models.DecimalField(decimal_places=2, max_digits=5)
  # objects = models.GeoManager()	# Disabled on purpose

Open a django shell and populate the model:

from demo.models import Demo
from django.contrib.gis.db.models import *

Demo(number="13.22").save()

A simple query

>>> Demo.objects.all().values()
[{'timestamp': datetime.datetime(2011, 7, 4, 16, 8, 26, 297486), 'id': 1, 'number': Decimal('13.22')}]

So far, so good

>>> Demo.objects.aggregate(Min("number"), Min("timestamp"))
{'number__min': Decimal('13.22'), 'timestamp__min': datetime.datetime(2011, 7, 4, 16, 8, 26, 297486)}

Just as expected.

Let's do it again, this time enabling the GeoManager:

from django.contrib.gis.db import models
class Demo(models.Model):
  timestamp = models.DateTimeField(auto_now_add=True)
  number = models.DecimalField(decimal_places=2, max_digits=5)
  objects = models.GeoManager()
from demo.models import Demo
from django.contrib.gis.db.models import *
>>> Demo.objects.all().values()
[{'timestamp': datetime.datetime(2011, 7, 4, 16, 8, 26, 297486), 'id': 1, 'number': Decimal('13.22')}]

The same...

Now...

>>> Demo.objects.aggregate(Min("number"), Min("timestamp"))
{'number__min': 13.220000000000001, 'timestamp__min': u'2011-07-04 16:08:26.297486'}

Whoops! float and string instead of Decimal and datetime!
See the difference? With GeoManager. convert_values() seems not to be called, at least for DateQuerySets and aggregates.

Attachments (4)

16408.1.diff (5.5 KB) - added by jbronn 3 years ago.
16408.2.diff (5.5 KB) - added by jbronn 3 years ago.
16408.3.diff (10.0 KB) - added by jbronn 3 years ago.
16408-4.diff (1.7 KB) - added by claudep 16 months ago.
Call parent convert_values

Download all attachments as: .zip

Change History (14)

comment:1 Changed 3 years ago by aaugustin

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

I didn't compile spatialite to confirm the bug, but the report is sufficiently detailed to accept the ticket.

This doesn't look related to #15040 and #15169.

comment:2 Changed 3 years ago by jbronn

  • Has patch set
  • milestone set to 1.4
  • Owner changed from nobody to jbronn
  • Status changed from new to assigned

Changed 3 years ago by jbronn

Changed 3 years ago by jbronn

Changed 3 years ago by jbronn

comment:3 Changed 3 years ago by jbronn

  • Resolution set to fixed
  • Status changed from assigned to closed

In [16749]:

Fixed #16408 -- Fixed conversion of dates, and other problems with the SpatiaLite backend.

comment:4 Changed 3 years ago by jbronn

In [16751]:

[1.3.X] Fixed #16408 -- Fixed conversion of dates, and other problems with the SpatiaLite backend.

Backport of r16749 and r16750 from trunk.

comment:5 follow-up: Changed 3 years ago by jacob

  • milestone 1.4 deleted

Milestone 1.4 deleted

comment:6 in reply to: ↑ 5 Changed 16 months ago by mathijs

  • Keywords regression added
  • Resolution fixed deleted
  • Status changed from closed to reopened
  • Version changed from 1.3 to 1.5-beta-1

This issue has re-appeared on django 1.5 b2.

I checked the source files and it appears the spatialite backend is not patched for this issue in this version, and is still using compilers etc. from django.contrib.gis.db.models.sql.compile instead of django.contrib.gis.db.models.spatialite.compile (which is missing)
merging the previous commits would solve it I guess...

I stand corrected, this seems to have moved down, outside the spatialite backend into the base sql backend, my appologies. I can correctly get the values as datatime objects in a shell.
The bug occurs when using the 'date_hierarchy' on an admin model, checking revealed the aggregate function still does not seem to call convert_values()

Last edited 16 months ago by mathijs (previous) (diff)

comment:7 Changed 16 months ago by mathijs

I should probably add the error occurs here:
/usr/local/lib/python2.7/dist-packages/django/contrib/admin/templatetags/admin_list.py in date_hierarchy, line 310

And that the problem disappears if I remove the objects = models.GeoManager() line from my model.

Changed 16 months ago by claudep

Call parent convert_values

comment:8 Changed 16 months ago by claudep

Could you please test the attached patch?

comment:9 Changed 16 months ago by Claude Paroz <claude@…>

  • Resolution set to fixed
  • Status changed from reopened to closed

In 0907d3c6f511ef3dc68d381389a32b9b54ae4be7:

Fixed #16408 -- Re-fixed value conversion with Spatialite backend

comment:10 Changed 16 months ago by Claude Paroz <claude@…>

In 4f67ab63761a949a4ee8dab5ae38f23b32dc56e5:

[1.5.x] Fixed #16408 -- Re-fixed value conversion with Spatialite backend

Backport of 0907d3c6f from master.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.