Opened 2 years ago

Closed 17 months ago

Last modified 17 months ago

#19678 closed Bug (fixed)

Test failure with spatialite

Reported by: julien Owned by: nobody
Component: GIS Version: 1.4
Severity: Normal Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I'm seeing a test failure with spatialite 4.0.0:

vagrant@djangocore:~$ runtests2.6-spatialite gis
Creating test database for alias 'default'...
Creating test database for alias 'other'...
.........................F.....................................................................................................................................................
======================================================================
FAIL: test02_osr (django.contrib.gis.tests.test_spatialrefsys.SpatialRefSysTest)
Testing getting OSR objects from SpatialRefSys model objects.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/django/django/contrib/gis/tests/test_spatialrefsys.py", line 76, in test02_osr
    self.assertTrue(srs.proj4 in sd['proj4'])
AssertionError: False is not True

----------------------------------------------------------------------
Ran 175 tests in 0.697s

FAILED (failures=1)
Destroying test database for alias 'default'...
Destroying test database for alias 'other'...

All the version numbers can be found in the latest djangocore-box specs: https://github.com/jphalip/djangocore-box/blob/master/provisioning/shell/install-gis.sh

Attachments (2)

19768-1.diff (2.2 KB) - added by claudep 22 months ago.
19678-2.diff (2.1 KB) - added by ramiro 17 months ago.

Download all attachments as: .zip

Change History (15)

comment:1 Changed 2 years ago by julien

  • Severity changed from Normal to Release blocker
  • Triage Stage changed from Unreviewed to Accepted

Marking as blocker to be sure 1.5 doesn't ship with a test failure.

comment:2 Changed 2 years ago by claudep

Julien, could you replace the assertTrue by an assertIn so as we can plainly see the strings involved in the comparison?

comment:3 Changed 2 years ago by julien

Hi Claude, this is the updated error message:

======================================================================
FAIL: test02_osr (django.contrib.gis.tests.test_spatialrefsys.SpatialRefSysTest)
Testing getting OSR objects from SpatialRefSys model objects.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/django/django/contrib/gis/tests/test_spatialrefsys.py", line 76, in test02_osr
    self.assertIn(srs.proj4, sd['proj4'])
AssertionError: u'+proj=longlat +ellps=WGS84 +towgs84=0,0,0,0,0,0,0 +no_defs ' not found in ['+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs ', '+proj=longlat +datum=WGS84 +no_defs ']

comment:4 Changed 2 years ago by claudep

That's strange. When reading libspatialite 4.0.0 sources, I can see the proper definition of the 4326 epsg definition (src/srsinit/epsg_inlined_wgs84_00.c):

p = add_epsg_def (filter, first, last, 4326, "epsg", 4326,
        "WGS 84");
add_proj4text (p, 0,
        "+proj=longlat +datum=WGS84 +no_defs");

What's the result of:
SELECT * FROM spatial_ref_sys WHERE srid=4326;

comment:5 Changed 2 years ago by julien

This is what I get when running that query:

[(4326, u'epsg', 4326, u'WGS 84', u'+proj=longlat +datum=WGS84 +no_defs', u'GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]]')]

comment:6 Changed 2 years ago by claudep

I've just compiled libspatialite 4.0.0 and ran the gis tests without errors. I have really no clue what your failure comes from.

comment:7 Changed 2 years ago by claudep

  • Severity changed from Release blocker to Normal

Quoting Anssi:
+1 for not holding 1.5 for this one. Spatialite isn't exactly the most used DB for gis, and the failure doesn't look like data corruption or crashing type.

Changed 22 months ago by claudep

comment:8 Changed 22 months ago by claudep

  • Has patch set

I've finally been able to reproduce the issue after upgrading my main system to Debian wheezy (using Spatialite 3). This has nothing to do with spatial_ref_sys data in Spatialite, but rather a consequence of the way the gdal.SpatialReference is initialized with a pseudo srs WKT.

Julien, would you be able to test the attached patch?

comment:9 Changed 21 months ago by timo

I can reproduce this using djangocore-box:

vagrant@djangocore:~$ runtests2.7-spatialite django.contrib.gis
Creating test database for alias 'default'...
Creating test database for alias 'other'...
..s..................sssssssssss................ssssss......................................................................................F................................Esssss.......................................................................................
======================================================================
ERROR: django.contrib.gis.tests.geoapp.tests (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: django.contrib.gis.tests.geoapp.tests
Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/loader.py", line 252, in _find_tests
    module = self._get_module_from_name(name)
  File "/usr/lib/python2.7/unittest/loader.py", line 230, in _get_module_from_name
    __import__(name)
  File "/django/django/contrib/gis/tests/geoapp/tests.py", line 207, in <module>
    class GeoLookupTest(TestCase):
  File "/django/django/contrib/gis/tests/geoapp/tests.py", line 307, in GeoLookupTest
    if postgis_bug_version():
  File "/django/django/contrib/gis/tests/geoapp/tests.py", line 27, in postgis_bug_version
    spatial_version = getattr(connection.ops, "spatial_version", (0,0,0))
  File "/django/django/utils/functional.py", line 49, in __get__
    res = instance.__dict__[self.func.__name__] = self.func(instance)
  File "/django/django/contrib/gis/db/backends/spatialite/operations.py", line 133, in spatial_version
    six.reraise(ImproperlyConfigured, ImproperlyConfigured(new_msg), sys.exc_info()[2])
  File "/django/django/contrib/gis/db/backends/spatialite/operations.py", line 127, in spatial_version
    version = self.spatialite_version_tuple()[1:]
  File "/django/django/contrib/gis/db/backends/spatialite/operations.py", line 273, in spatialite_version_tuple
    version = self.spatialite_version()
  File "/django/django/contrib/gis/db/backends/spatialite/operations.py", line 264, in spatialite_version
    return self._get_spatialite_func('spatialite_version()')
  File "/django/django/contrib/gis/db/backends/spatialite/operations.py", line 242, in _get_spatialite_func
    cursor = self.connection._cursor()
  File "/django/django/db/backends/__init__.py", line 131, in _cursor
    self.ensure_connection()
  File "/django/django/db/backends/__init__.py", line 126, in ensure_connection
    self.connect()
  File "/django/django/db/backends/__init__.py", line 113, in connect
    conn_params = self.get_connection_params()
  File "/django/django/db/backends/sqlite3/base.py", line 322, in get_connection_params
    "settings.DATABASES is improperly configured. "
ImproperlyConfigured: Cannot determine the SpatiaLite version for the "" database (error was "settings.DATABASES is improperly configured. Please supply the NAME value.").  Was the SpatiaLite initialization SQL loaded on this database?


======================================================================
FAIL: test02_osr (django.contrib.gis.tests.test_spatialrefsys.SpatialRefSysTest)
Testing getting OSR objects from SpatialRefSys model objects.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/django/django/contrib/gis/tests/test_spatialrefsys.py", line 77, in test02_osr
    six.assertRegex(self, srs.proj4, sd['proj4_re'])
  File "/django/django/utils/six.py", line 430, in assertRegex
    return getattr(self, _assertRegex)(*args, **kwargs)
AssertionError: Regexp didn't match: '\\+proj=longlat (\\+ellps=WGS84 )?\\+datum=WGS84 \\+no_defs ' not found in u'+proj=longlat +ellps=WGS84 +towgs84=0,0,0,0,0,0,0 +no_defs '

----------------------------------------------------------------------
Ran 266 tests in 2.289s

FAILED (failures=1, errors=1, skipped=23)

After the patch is applied, none of the GIS tests run:

vagrant@djangocore:~$ runtests2.7-spatialite django.contrib.gis
Traceback (most recent call last):
  File "/django/tests/runtests.py", line 364, in <module>
    options.failfast, args)
  File "/django/tests/runtests.py", line 193, in django_tests
    state = setup(verbosity, test_labels)
  File "/django/tests/runtests.py", line 140, in setup
    test_modules = get_test_modules()
  File "/django/tests/runtests.py", line 67, in get_test_modules
    from django.contrib.gis.tests.utils import HAS_SPATIAL_DB
  File "/django/django/contrib/gis/tests/utils.py", line 34, in <module>
    from django.contrib.gis.db.backends.spatialite.models import SpatialRefSys
  File "/django/django/contrib/gis/db/backends/spatialite/models.py", line 45, in <module>
    class SpatialRefSys(models.Model, SpatialRefSysMixin):
  File "/django/django/contrib/gis/db/backends/spatialite/models.py", line 54, in SpatialRefSys
    if connection.ops.spatial_version[0] >= 3:
  File "/django/django/utils/functional.py", line 49, in __get__
    res = instance.__dict__[self.func.__name__] = self.func(instance)
  File "/django/django/contrib/gis/db/backends/spatialite/operations.py", line 133, in spatial_version
    six.reraise(ImproperlyConfigured, ImproperlyConfigured(new_msg), sys.exc_info()[2])
  File "/django/django/contrib/gis/db/backends/spatialite/operations.py", line 127, in spatial_version
    version = self.spatialite_version_tuple()[1:]
  File "/django/django/contrib/gis/db/backends/spatialite/operations.py", line 273, in spatialite_version_tuple
    version = self.spatialite_version()
  File "/django/django/contrib/gis/db/backends/spatialite/operations.py", line 264, in spatialite_version
    return self._get_spatialite_func('spatialite_version()')
  File "/django/django/contrib/gis/db/backends/spatialite/operations.py", line 242, in _get_spatialite_func
    cursor = self.connection._cursor()
  File "/django/django/db/backends/__init__.py", line 131, in _cursor
    self.ensure_connection()
  File "/django/django/db/backends/__init__.py", line 126, in ensure_connection
    self.connect()
  File "/django/django/db/backends/__init__.py", line 113, in connect
    conn_params = self.get_connection_params()
  File "/django/django/db/backends/sqlite3/base.py", line 322, in get_connection_params
    "settings.DATABASES is improperly configured. "
django.core.exceptions.ImproperlyConfigured: Cannot determine the SpatiaLite version for the "" database (error was "settings.DATABASES is improperly configured. Please supply the NAME value.").  Was the SpatiaLite initialization SQL loaded on this database?

Changed 17 months ago by ramiro

comment:10 Changed 17 months ago by ramiro

I've uploaded a modified version of Claude's patch that works for me (spatialite 3.x here, note how the version conditional was changed to only in version 4.x otherwise most of the GIS tests fail becuase AFAICS libspatialite 3 doesn't create the new srtex column).

Perhaps this can be divided in two different commits?

Version 0, edited 17 months ago by ramiro (next)

comment:11 Changed 17 months ago by claudep

  • Triage Stage changed from Accepted to Ready for checkin

Ramiro, please go ahead. I can confirm that the srtext column is 4.x only (was previously named srs_wkt, but not used in GeoDjango).

comment:12 Changed 17 months ago by Ramiro Morales <cramm0@…>

  • Resolution set to fixed
  • Status changed from new to closed

In 9a73e7f40c851cef262dd36da444c81492632ee5:

Fixed #19678 -- GeoDjango test failure with spatialite >= 3.0.

Thanks Julien for the report and Claude for the fix.

comment:13 Changed 17 months ago by Ramiro Morales <cramm0@…>

In 362dd68fb20be195462af22448416c9895ce7df7:

Added new 'srtext' spatialite 4.x SpatialRefSys column to its model.

This is for general consistency in the GeoDjango DB backends.
Thanks Claude for the fix.

Refs #19678.

Note: See TracTickets for help on using tickets.
Back to Top