Query result types not being converted with Spatialite

[Antonio Eugenio Burriel <aeburriel@…>]

While making a model gis-aware, I've noticed the following wrong behavior with the Spatialite backend (works as expected with Postgis).

Consider the following model:

from django.contrib.gis.db import models
class Demo(models.Model):
  timestamp = models.DateTimeField(auto_now_add=True)
  number = models.DecimalField(decimal_places=2, max_digits=5)
  # objects = models.GeoManager()	# Disabled on purpose

Open a django shell and populate the model:

from demo.models import Demo
from django.contrib.gis.db.models import *

Demo(number="13.22").save()

A simple query

>>> Demo.objects.all().values()
[{'timestamp': datetime.datetime(2011, 7, 4, 16, 8, 26, 297486), 'id': 1, 'number': Decimal('13.22')}]

So far, so good

>>> Demo.objects.aggregate(Min("number"), Min("timestamp"))
{'number__min': Decimal('13.22'), 'timestamp__min': datetime.datetime(2011, 7, 4, 16, 8, 26, 297486)}

Just as expected.

Let's do it again, this time enabling the GeoManager:

from django.contrib.gis.db import models
class Demo(models.Model):
  timestamp = models.DateTimeField(auto_now_add=True)
  number = models.DecimalField(decimal_places=2, max_digits=5)
  objects = models.GeoManager()

from demo.models import Demo
from django.contrib.gis.db.models import *

>>> Demo.objects.all().values()
[{'timestamp': datetime.datetime(2011, 7, 4, 16, 8, 26, 297486), 'id': 1, 'number': Decimal('13.22')}]

The same...

Now...

>>> Demo.objects.aggregate(Min("number"), Min("timestamp"))
{'number__min': 13.220000000000001, 'timestamp__min': u'2011-07-04 16:08:26.297486'}

Whoops! float and string instead of Decimal and datetime!
See the difference? With GeoManager. convert_values() seems not to be called, at least for DateQuerySets and aggregates.

[Aymeric Augustin]

I didn't compile spatialite to confirm the bug, but the report is sufficiently detailed to accept the ticket.

This doesn't look related to #15040 and #15169.

[jbronn]

In [16749]:

Fixed #16408 -- Fixed conversion of dates, and other problems with the SpatiaLite backend.

[jbronn]

In [16751]:

[1.3.X] Fixed #16408 -- Fixed conversion of dates, and other problems with the SpatiaLite backend.

Backport of r16749 and r16750 from trunk.

[Jacob]

Milestone 1.4 deleted

[Mathijs Dumon]

This issue has re-appeared on django 1.5 b2.

I checked the source files and it appears the spatialite backend is not patched for this issue in this version, and is still using compilers etc. from django.contrib.gis.db.models.sql.compile instead of django.contrib.gis.db.models.spatialite.compile (which is missing)
merging the previous commits would solve it I guess...
I stand corrected, this seems to have moved down, outside the spatialite backend into the base sql backend, my appologies. I can correctly get the values as datatime objects in a shell.
The bug occurs when using the 'date_hierarchy' on an admin model

[Mathijs Dumon]

I should probably add the error occurs here:
/usr/local/lib/python2.7/dist-packages/django/contrib/admin/templatetags/admin_list.py in date_hierarchy, line 310

And that the problem disappears if I remove the objects = models.GeoManager() line from my model.

[Claude Paroz]

Call parent convert_values

[Claude Paroz]

Could you please test the attached patch?

[Claude Paroz <claude@…>]

In 0907d3c6f511ef3dc68d381389a32b9b54ae4be7:

Fixed #16408 -- Re-fixed value conversion with Spatialite backend

[Claude Paroz <claude@…>]

In 4f67ab63761a949a4ee8dab5ae38f23b32dc56e5:

[1.5.x] Fixed #16408 -- Re-fixed value conversion with Spatialite backend

Backport of 0907d3c6f from master.