Opened 7 years ago

Closed 4 years ago

Last modified 4 years ago

#27399 closed Bug (needsinfo)

Segfault when running GDALBandTests on OSX 10.11.6

Reported by: Tatiana Vasilevskaya Owned by: nobody
Component: GIS Version: dev
Severity: Normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

When running gis_tests.gdal_tests.test_raster.GDALBandTests tests on OSX 10.11.6 django calls capi.open_ds inside GDALRaster. This results in a Segmentation fault. Occurs on the master branch (80e742d991b276023a3aa51a29ed757879051282).

  /Users/vasilty/Documents/Projects/django/tests/runtests.py(483)<module>()
-> options.exclude_tags,
  /Users/vasilty/Documents/Projects/django/tests/runtests.py(277)django_tests()
-> extra_tests=extra_tests,
  /Users/vasilty/Documents/Projects/django/django/test/runner.py(596)run_tests()
-> result = self.run_suite(suite)
  /Users/vasilty/Documents/Projects/django/django/test/runner.py(561)run_suite()
-> return runner.run(suite)
  /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/runner.py(176)run()
-> test(result)
  /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/suite.py(84)__call__()
-> return self.run(*args, **kwds)
  /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/suite.py(122)run()
-> test(result)
  /Users/vasilty/Documents/Projects/django/django/test/testcases.py(211)__call__()
-> super(SimpleTestCase, self).__call__(result)
  /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/case.py(648)__call__()
-> return self.run(*args, **kwds)
  /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/case.py(596)run()
-> self.setUp()
> /Users/vasilty/Documents/Projects/django/tests/gis_tests/gdal_tests/test_raster.py(312)setUp()
-> self.rs_path = os.path.join(os.path.dirname(upath(__file__)), '../data/rasters/raster.tif')

TIFFReadDirectory: Warning, Unknown field with tag 42113 (0xa481) encountered.
Segmentation fault: 11

Change History (9)

comment:1 by Markus Holtermann, 7 years ago

Triage Stage: UnreviewedAccepted

comment:2 by Tim Graham, 7 years ago

Do you have any details about why Django might be at fault? We don't ship any C code, so segfaults are generally not caused by Django.

comment:3 by Markus Holtermann, 7 years ago

Even if we don't ship C code, either our documentation lacks some details on supported GDAL / GIS versions (that's beyond my knowledge) or we should check for that. Either way, I don't think Django's test should segfault regardless.

comment:4 by Tim Graham, 7 years ago

What are the installed GIS library versions? Did you reproduce the issue, Markus?

Last edited 7 years ago by Tim Graham (previous) (diff)

comment:5 by Saulius Žemaitaitis, 7 years ago

I can still reproduce it on master (24c9cce1c9fb0dfe14e42de1ec2a2a209a64dd80).

OSX 10.11.6

Library versions (installed from Homebrew):

gdal 1.11.5_1
geos 3.5.0
libgeotiff 1.4.1_2
liblwgeom 2.1.5_3
libtiff 4.0.6_2

GDB backtrace:

#0  0x000000010308f0b6 in _TIFFVGetField () from /Users/saulius/sandbox/duth2016/django/env/lib/python3.5/site-packages/PIL/.dylibs/libtiff.5.dylib
#1  0x0000000105332010 in TIFFGetField () from /usr/local/opt/libtiff/lib/libtiff.5.dylib
#2  0x0000000104a21b61 in GTiffDataset::OpenOffset(tiff*, GTiffDataset**, unsigned long, int, GDALAccess, int, int, char**) () from /usr/local/lib/libgdal.dylib
#3  0x0000000104a24a64 in GTiffDataset::Open(GDALOpenInfo*) () from /usr/local/lib/libgdal.dylib
#4  0x0000000104b956d9 in GDALOpenInternal(GDALOpenInfo&, char const* const*) () from /usr/local/lib/libgdal.dylib
#5  0x0000000104b95580 in GDALOpenInternal(char const*, GDALAccess, char const* const*) () from /usr/local/lib/libgdal.dylib
#6  0x0000000102e3f79f in ffi_call_unix64 () from /Users/saulius/sandbox/duth2016/django/env/lib/python3.5/lib-dynload/_ctypes.cpython-35m-darwin.so
#7  0x0000000102e3ffd8 in ffi_call () from /Users/saulius/sandbox/duth2016/django/env/lib/python3.5/lib-dynload/_ctypes.cpython-35m-darwin.so
#8  0x0000000102e3b89a in _ctypes_callproc () from /Users/saulius/sandbox/duth2016/django/env/lib/python3.5/lib-dynload/_ctypes.cpython-35m-darwin.so
#9  0x0000000102e36473 in PyCFuncPtr_call () from /Users/saulius/sandbox/duth2016/django/env/lib/python3.5/lib-dynload/_ctypes.cpython-35m-darwin.so
#10 0x000000010000ecb5 in PyObject_Call () from /usr/local/Cellar/python3/3.5.2_2/Frameworks/Python.framework/Versions/3.5/Python
Last edited 7 years ago by Saulius Žemaitaitis (previous) (diff)

comment:6 by Saulius Žemaitaitis, 7 years ago

The problem seems to be due to the way Pillow and GDAL interact. Pillow wheel ships its own version of libtiff while GDAL library uses the system one. This apparently causes the segfault. I don't have the skills to narrow it down further.

Workaround is to install Pillow from source on OSX:

pip install --no-binary Pillow -r tests/requirements/py3.txt

comment:7 by winsent, 6 years ago

Look at this example. Install only pillow and Django without GDAL python binding. This code raise exeption after pillow installed.

GEOTIF_PATH=/ta/dem.tif

virtualenv --python=python3 test
cd test
source bin/activate
pip install django==1.11.9 pillow
python -c "from django.contrib.gis.gdal import GDALRaster; print(GDALRaster('$GEOTIF_PATH'))"
django-admin startproject testtest
cd testtest
django-admin startapp testtesttest
echo "\n\nINSTALLED_APPS += ['testtesttest']" >> testtest/settings.py
mkdir -p testtesttest/management/commands
touch testtesttest/management/__init__.py testtesttest/management/commands/__init__.py
echo "from django.core.management.base import BaseCommand
class Command(BaseCommand):
    def handle(self, *args, **options):
        from django.contrib.gis.gdal import GDALRaster
        print(GDALRaster('$GEOTIF_PATH'))
" > testtesttest/management/commands/gdal_test.py
python manage.py gdal_test
pip uninstall -y pillow
python manage.py gdal_test

Result:

TIFFReadDirectory: Warning, Unknown field with tag 42113 (0xa481) encountered.
[1]    53998 bus error  python manage.py gdal_test

comment:8 by Mariusz Felisiak, 4 years ago

Resolution: invalid
Status: newclosed
Triage Stage: AcceptedUnreviewed

Django 1.11 and GDAL 1.11 are no longer supported. Closing unless someone can provide a clear path to reproduce this issue on supported versions.

comment:9 by Mariusz Felisiak, 4 years ago

Resolution: invalidneedsinfo
Note: See TracTickets for help on using tickets.
Back to Top