GeoDjango with Django 1.8 - 'GDALAllRegister' Attribute Error

[dimitri-justeau]

I used to work with geodjango using Django 1.7 on Windows 7, and everything was working well. After having upgraded to django 1.8, loading the gis app causes an error.

Here is the full Traceback:

Traceback (most recent call last):
  File "C:\Users\justeau\PycharmProjects\Digitizing-Monitor\digitizing_monitor\manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "C:\Python34\lib\site-packages\django\core\management\__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "C:\Python34\lib\site-packages\django\core\management\__init__.py", line 312, in execute
    django.setup()
  File "C:\Python34\lib\site-packages\django\__init__.py", line 18, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "C:\Python34\lib\site-packages\django\apps\registry.py", line 108, in populate
    app_config.import_models(all_models)
  File "C:\Python34\lib\site-packages\django\apps\config.py", line 198, in import_models
    self.models_module = import_module(models_module_name)
  File "C:\Python34\lib\importlib\__init__.py", line 109, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "<frozen importlib._bootstrap>", line 1471, in exec_module
  File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
  File "C:\Python34\lib\site-packages\django\contrib\auth\models.py", line 41, in <module>
    class Permission(models.Model):
  File "C:\Python34\lib\site-packages\django\db\models\base.py", line 139, in __new__
    new_class.add_to_class('_meta', Options(meta, **kwargs))
  File "C:\Python34\lib\site-packages\django\db\models\base.py", line 324, in add_to_class
    value.contribute_to_class(cls, name)
  File "C:\Python34\lib\site-packages\django\db\models\options.py", line 250, in contribute_to_class
    self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
  File "C:\Python34\lib\site-packages\django\db\__init__.py", line 36, in __getattr__
    return getattr(connections[DEFAULT_DB_ALIAS], item)
  File "C:\Python34\lib\site-packages\django\db\utils.py", line 240, in __getitem__
    backend = load_backend(db['ENGINE'])
  File "C:\Python34\lib\site-packages\django\db\utils.py", line 111, in load_backend
    return import_module('%s.base' % backend_name)
  File "C:\Python34\lib\importlib\__init__.py", line 109, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "<frozen importlib._bootstrap>", line 1471, in exec_module
  File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
  File "C:\Python34\lib\site-packages\django\contrib\gis\db\backends\postgis\base.py", line 8, in <module>
    from .features import DatabaseFeatures
  File "C:\Python34\lib\site-packages\django\contrib\gis\db\backends\postgis\features.py", line 1, in <module>
    from django.contrib.gis.db.backends.base.features import BaseSpatialFeatures
  File "C:\Python34\lib\site-packages\django\contrib\gis\db\backends\base\features.py", line 3, in <module>
    from django.contrib.gis.db.models import aggregates
  File "C:\Python34\lib\site-packages\django\contrib\gis\db\models\__init__.py", line 7, in <module>
    from django.contrib.gis.geos import HAS_GEOS
  File "C:\Python34\lib\site-packages\django\contrib\gis\geos\__init__.py", line 16, in <module>
    from .geometry import GEOSGeometry, wkt_regex, hex_regex
  File "C:\Python34\lib\site-packages\django\contrib\gis\geos\geometry.py", line 10, in <module>
    from django.contrib.gis.gdal.error import SRSException
  File "C:\Python34\lib\site-packages\django\contrib\gis\gdal\__init__.py", line 47, in <module>
    from django.contrib.gis.gdal.driver import Driver  # NOQA
  File "C:\Python34\lib\site-packages\django\contrib\gis\gdal\driver.py", line 5, in <module>
    from django.contrib.gis.gdal.prototypes import ds as vcapi, raster as rcapi
  File "C:\Python34\lib\site-packages\django\contrib\gis\gdal\prototypes\raster.py", line 25, in <module>
    register_all = void_output(lgdal.GDALAllRegister, [])
  File "C:\Python34\lib\ctypes\__init__.py", line 364, in __getattr__
    func = self.__getitem__(name)
  File "C:\Python34\lib\ctypes\__init__.py", line 369, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: function 'GDALAllRegister' not found

After some searches, it appears that other people had this problem when using GeoDjango + Django 1.8 + Windows (example: ​https://gis.stackexchange.com/questions/143110/geodjango-setup-python-gdalallregister-attribute-error/144407#144407). But I could not find any fix for this.

I investigated a bit, and found out that at django.contrib.gis.gdal.libgal, we can see this:

    ..|
    47|  # This loads the GDAL/OGR C library
    48|  lgdal = CDLL(lib_path)
    49| 
    50|  # On Windows, the GDAL binaries have some OSR routines exported with
    51|  # STDCALL, while others are not.  Thus, the library will also need to
    52|  # be loaded up as WinDLL for said OSR functions that require the
    53|  # different calling convention.
    54|  if os.name == 'nt':
    55|      from ctypes import WinDLL
    56|    lwingdal = WinDLL(lib_path)
    ..|

So when the os is Windows, the GDAL library is also loaded as a DLL, so we have two GDAL libraries, lgdal and lwingdal.

But if we look at the django.contrib.gis.gdal.prototypes.raster (where the "'GDALAllRegister' not found" error is raised from), we can see that the one used is lgdal:

    ..|
    29|  # Raster Driver Routines
    30|  register_all = void_output(lgdal.GDALAllRegister, [])
    31|  get_driver = voidptr_output(lgdal.GDALGetDriver, [c_int])
    32|  get_driver_by_name = voidptr_output(lgdal.GDALGetDriverByName, [c_char_p], errcheck=False)
    33|  get_driver_count = int_output(lgdal.GDALGetDriverCount, [])
    34|  get_driver_description = const_string_output(lgdal.GDALGetDescription, [c_void_p])
    ..|

It seems like on Windows, lwingdal should be used, so I added this after the imports:

    ..|  
    15|  # Fix for windows users
    16|  if os.name == 'nt':
    17|      from django.contrib.gis.gdal.libgdal import lwingdal 
    18|      lgdal = lwingdal
    ..| 

And it solved the problem for me! Maybe there is something more, it seems a bit too easy, but I still had no issue with it, and I now can use the GIS with Django 1.8 on Windows, which was impossible before this.

Hope I helped...

Regards,

Dimitri.

[Tim Graham]

Created a ​PR based on the proposed fix. Going to try to see if I can get the GIS tests running on Windows to confirm the fix.

[Tim Graham]

After applying that patch, I have the following test failures on my Windows system. Some of them may have preceded the work on rasters, but the raster failures are a bit troubling as far as confirming this patch is correct. Do you have any expertise to help resolve them? GDAL version is 1.11.2.

======================================================================
ERROR: test03a_layers (gis_tests.gdal_tests.test_ds.DataSourceTest)
Testing Data Source Layers.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Tim\code\django\tests\gis_tests\gdal_tests\test_ds.py", line 12
5, in test03a_layers
    self.assertRaises(OGRIndexError, layer.__getitem__, 50000)
  File "C:\Python27\Lib\unittest\case.py", line 473, in assertRaises
    callableObj(*args, **kwargs)
  File "c:\users\tim\code\django\django\contrib\gis\gdal\layer.py", line 51, in
__getitem__
    return self._make_feature(index)
  File "c:\users\tim\code\django\django\contrib\gis\gdal\layer.py", line 84, in
_make_feature
    return Feature(capi.get_feature(self.ptr, feat_id), self)
WindowsError: [Error -1073741795] Windows Error 0xC000001D

======================================================================
ERROR: test07b_closepolygons (gis_tests.gdal_tests.test_geom.OGRGeomTest)
Testing closing Polygon objects.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Tim\code\django\tests\gis_tests\gdal_tests\test_geom.py", line
250, in test07b_closepolygons
    poly.centroid
  File "c:\users\tim\code\django\django\contrib\gis\gdal\geometries.py", line 62
7, in centroid
    capi.get_centroid(self.ptr, p.ptr)
WindowsError: [Error -1073741795] Windows Error 0xC000001D

======================================================================
ERROR: test_band_data_setters (gis_tests.gdal_tests.test_raster.GDALBandTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Tim\code\django\tests\gis_tests\gdal_tests\test_raster.py", lin
e 194, in test_band_data_setters
    'srid': 4326,
  File "c:\users\tim\code\django\django\contrib\gis\gdal\raster\source.py", line
 105, in __init__
    None
TypeError: this function takes 6 arguments (7 given)

======================================================================
ERROR: test_file_based_raster_creation (gis_tests.gdal_tests.test_raster.GDALRas
terTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Tim\code\django\tests\gis_tests\gdal_tests\test_raster.py", lin
e 142, in test_file_based_raster_creation
    'nodata_value': self.rs.bands[0].nodata_value
  File "c:\users\tim\code\django\django\contrib\gis\gdal\raster\source.py", line
 105, in __init__
    None
TypeError: this function takes 6 arguments (7 given)

======================================================================
ERROR: test_geotransform_and_friends (gis_tests.gdal_tests.test_raster.GDALRaste
rTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Tim\code\django\tests\gis_tests\gdal_tests\test_raster.py", lin
e 100, in test_geotransform_and_friends
    rsmem = GDALRaster(JSON_RASTER)
  File "c:\users\tim\code\django\django\contrib\gis\gdal\raster\source.py", line
 105, in __init__
    None
TypeError: this function takes 6 arguments (7 given)

======================================================================
ERROR: test_custom_srid (gis_tests.geos_tests.test_geos.GEOSTest)
Test with a srid unknown from GDAL
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Tim\code\django\tests\gis_tests\geos_tests\test_geos.py", line
647, in test_custom_srid
    self.assertIsInstance(pnt.ogr, gdal.OGRGeometry)
  File "c:\users\tim\code\django\django\contrib\gis\geos\geometry.py", line 462,
 in ogr
    return gdal.OGRGeometry(self.wkb, self.srid)
  File "c:\users\tim\code\django\django\contrib\gis\gdal\geometries.py", line 11
7, in __init__
    self.srs = srs
  File "c:\users\tim\code\django\django\contrib\gis\gdal\geometries.py", line 27
1, in _set_srs
    sr = SpatialReference(srs)
  File "c:\users\tim\code\django\django\contrib\gis\gdal\srs.py", line 95, in __
init__
    self.import_epsg(srs_input)
  File "c:\users\tim\code\django\django\contrib\gis\gdal\srs.py", line 286, in i
mport_epsg
    capi.from_epsg(self.ptr, epsg)
WindowsError: exception: access violation writing 0x0000000F

[Tim Graham]

I managed to fix some of the failures. The remaining failures are below (not sure if they are Django problems that need to be fixed or a problem with my test machine setup).

======================================================================
ERROR: test03a_layers (gis_tests.gdal_tests.test_ds.DataSourceTest)
Testing Data Source Layers.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Tim\code\django\tests\gis_tests\gdal_tests\test_ds.py", line 12
5, in test03a_layers
    self.assertRaises(OGRIndexError, layer.__getitem__, 50000)
  File "C:\Program Files\Python 3.5\Lib\unittest\case.py", line 704, in assertRa
ises
    return context.handle('assertRaises', callableObj, args, kwargs)
  File "C:\Program Files\Python 3.5\Lib\unittest\case.py", line 162, in handle
    callable_obj(*args, **kwargs)
  File "c:\Users\Tim\code\django\django\contrib\gis\gdal\layer.py", line 51, in
__getitem__
    return self._make_feature(index)
  File "c:\Users\Tim\code\django\django\contrib\gis\gdal\layer.py", line 84, in
_make_feature
    return Feature(capi.get_feature(self.ptr, feat_id), self)
OSError: [WinError -1073741795] Windows Error 0xc000001d

======================================================================
ERROR: test07b_closepolygons (gis_tests.gdal_tests.test_geom.OGRGeomTest)
Testing closing Polygon objects.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Tim\code\django\tests\gis_tests\gdal_tests\test_geom.py", line
250, in test07b_closepolygons
    poly.centroid
  File "c:\Users\Tim\code\django\django\contrib\gis\gdal\geometries.py", line 62
7, in centroid
    capi.get_centroid(self.ptr, p.ptr)
OSError: [WinError -1073741795] Windows Error 0xc000001d

======================================================================
ERROR: test_file_based_raster_creation (gis_tests.gdal_tests.test_raster.GDALRas
terTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Tim\code\django\tests\gis_tests\gdal_tests\test_raster.py", lin
e 142, in test_file_based_raster_creation
    'nodata_value': self.rs.bands[0].nodata_value
  File "c:\Users\Tim\code\django\django\contrib\gis\gdal\raster\source.py", line
 105, in __init__
    None
OSError: [WinError -1073741795] Windows Error 0xc000001d

======================================================================
ERROR: test_custom_srid (gis_tests.geos_tests.test_geos.GEOSTest)
Test with a srid unknown from GDAL
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Tim\code\django\tests\gis_tests\geos_tests\test_geos.py", line
647, in test_custom_srid
    self.assertIsInstance(pnt.ogr, gdal.OGRGeometry)
  File "c:\Users\Tim\code\django\django\contrib\gis\geos\geometry.py", line 462,
 in ogr
    return gdal.OGRGeometry(self.wkb, self.srid)
  File "c:\Users\Tim\code\django\django\contrib\gis\gdal\geometries.py", line 11
7, in __init__
    self.srs = srs
  File "c:\Users\Tim\code\django\django\contrib\gis\gdal\geometries.py", line 27
1, in _set_srs
    sr = SpatialReference(srs)
  File "c:\Users\Tim\code\django\django\contrib\gis\gdal\srs.py", line 95, in __
init__
    self.import_epsg(srs_input)
  File "c:\Users\Tim\code\django\django\contrib\gis\gdal\srs.py", line 286, in i
mport_epsg
    capi.from_epsg(self.ptr, epsg)
OSError: exception: access violation writing 0x0000000F

[Tim Graham <timograham@…>]

In 3c4fca2:

Fixed #24724 -- Fixed GIS initialization crash on Windows.

[Tim Graham <timograham@…>]

In d7ea279f:

[1.8.x] Fixed #24724 -- Fixed GIS initialization crash on Windows.

Backport of 3c4fca271bd6850d252c4b637871e597484804d1 from master

[dimitri-justeau]

Hello,

Thank you for having fixed the issue! About the raster test failures, I have not the expertise to help with this since I'm mainly working with vectors, and not so much with rasters...

PS: Sorry for not having been more reactive, I did not received any notification about your comments.