gevent-monkeypatching with GIS PointField causes intermittent traceback on shutdown

[James Addison]

With the following model and a management command script that creates several instances of this model - some with point = None - I get a traceback on shutdown of the management command script. Note that the instances are created/saved to the database perfectly fine. I also am unable to reproduce without gevent.

from django.contrib.gis.db import models

class Provider(models.Model):
    point = models.PointField(blank=True, null=True)

Management script:

from django.contrib.gis.geos import Point
from django.core.management import BaseCommand
from businesses.models import Provider

class Command(BaseCommand):
    def handle(self, *args, **options):
        for i in range(100):
            Provider.objects.create(
                point=Point(-123.329773, 48.407326)
            )
            Provider.objects.create(
                point=None
            )

Finally, the traceback:

(geos_exception)vagrant@vagrant-ubuntu-trusty-64:/vagrant/geos_exception$ ./manage.py testcomm --settings=geos_exception.settings
Exception ignored in: <bound method WKBWriter.__del__ of <django.contrib.gis.geos.prototypes.io.WKBWriter object at 0x7f7b62d85198>>
Traceback (most recent call last):
  File "/home/vagrant/.virtualenvs/geos_exception/lib/python3.4/site-packages/django/contrib/gis/geos/prototypes/io.py", line 137, in __del__
  File "/home/vagrant/.virtualenvs/geos_exception/lib/python3.4/site-packages/django/contrib/gis/geos/libgeos.py", line 157, in __call__
  File "/home/vagrant/.virtualenvs/geos_exception/lib/python3.4/site-packages/django/contrib/gis/geos/prototypes/threadsafe.py", line 52, in __call__
  File "/home/vagrant/.virtualenvs/geos_exception/lib/python3.4/site-packages/gevent/local.py", line 246, in __getattribute__
TypeError: 'NoneType' object is not callable

I am attached a tar.gz file containing the test project. Remember, this issue is intermittent - I can run the management command a time or two without traceback before seeing the problem.

[James Addison]

Sample project exhibiting the problem.

[Claude Paroz]

All of this is related to garbage collection, that's clear. WKBWriter/ WKBReader instances have a __del__ method, so they are not part of the normal Python garbage collection cycle (read ​https://docs.python.org/3/reference/datamodel.html?highlight=__del__#object.__del).

Looking at the traceback though, it seems that the error is happening in gevent itself, that's why I doubt we'll able to fix this on Django's side. Also note that these errors are more annoying than harmful.

#20903 also reported this (but without reproducible code).

[James Addison]

New information: I have also seen this traceback, and I haven't even used the model with the PointField:

Exception ignored in: <bound method Point.__del__ of <Point object at 0x7fb1c9e55a28>>
Traceback (most recent call last):
  File "/home/vagrant/.virtualenvs/myproject/src/django/django/contrib/gis/geos/geometry.py", line 125, in __del__
  File "/home/vagrant/.virtualenvs/myproject/src/django/django/contrib/gis/geos/libgeos.py", line 156, in __call__
  File "/home/vagrant/.virtualenvs/myproject/src/django/django/contrib/gis/geos/libgeos.py", line 160, in get_func
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2222, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 2155, in _find_spec
TypeError: 'NoneType' object is not iterable

[Claude Paroz]

I also got one of those exception today with Django 1.8.5/Python 3.4.

I think we should simply catch TypeError in the destructor methods and ignore in case of exceptions.

So instead of current:

if self._ptr and capi:
        capi.release_srs(self._ptr)

Let's write:

try:
    capi.release_srs(self._ptr)
except TypeError:
    pass

[Claude Paroz]

​PR

[Claude Paroz <claude@…>]

In cd40d9e7:

Fixed #25657 -- Ignored exceptions when destroying geometry objects

Due to randomness of garbage collection with geometry objects, it's
easier to simply ignore AttributeError/TypeError generally raised when
parts of objects are already garbage-collected.
Thanks Sergey Fedoseev and Tim Graham for reviewing the patch.