Opened 3 years ago

Closed 3 years ago

#25657 closed Bug (fixed)

gevent-monkeypatching with GIS PointField causes intermittent traceback on shutdown

Reported by: James Addison Owned by: nobody
Component: GIS Version: 1.9
Severity: Normal Keywords: gevent
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

With the following model and a management command script that creates several instances of this model - some with point = None - I get a traceback on shutdown of the management command script. Note that the instances are created/saved to the database perfectly fine. I also am unable to reproduce without gevent.

from django.contrib.gis.db import models

class Provider(models.Model):
    point = models.PointField(blank=True, null=True)

Management script:

from django.contrib.gis.geos import Point
from django.core.management import BaseCommand
from businesses.models import Provider

class Command(BaseCommand):
    def handle(self, *args, **options):
        for i in range(100):
            Provider.objects.create(
                point=Point(-123.329773, 48.407326)
            )
            Provider.objects.create(
                point=None
            )

Finally, the traceback:

(geos_exception)vagrant@vagrant-ubuntu-trusty-64:/vagrant/geos_exception$ ./manage.py testcomm --settings=geos_exception.settings
Exception ignored in: <bound method WKBWriter.__del__ of <django.contrib.gis.geos.prototypes.io.WKBWriter object at 0x7f7b62d85198>>
Traceback (most recent call last):
  File "/home/vagrant/.virtualenvs/geos_exception/lib/python3.4/site-packages/django/contrib/gis/geos/prototypes/io.py", line 137, in __del__
  File "/home/vagrant/.virtualenvs/geos_exception/lib/python3.4/site-packages/django/contrib/gis/geos/libgeos.py", line 157, in __call__
  File "/home/vagrant/.virtualenvs/geos_exception/lib/python3.4/site-packages/django/contrib/gis/geos/prototypes/threadsafe.py", line 52, in __call__
  File "/home/vagrant/.virtualenvs/geos_exception/lib/python3.4/site-packages/gevent/local.py", line 246, in __getattribute__
TypeError: 'NoneType' object is not callable

I am attached a tar.gz file containing the test project. Remember, this issue is intermittent - I can run the management command a time or two without traceback before seeing the problem.

Attachments (1)

geos_exception.tar.gz (3.1 KB) - added by James Addison 3 years ago.
Sample project exhibiting the problem.

Download all attachments as: .zip

Change History (8)

Changed 3 years ago by James Addison

Attachment: geos_exception.tar.gz added

Sample project exhibiting the problem.

comment:1 Changed 3 years ago by Claude Paroz

All of this is related to garbage collection, that's clear. WKBWriter/ WKBReader instances have a __del__ method, so they are not part of the normal Python garbage collection cycle (read https://docs.python.org/3/reference/datamodel.html?highlight=__del__#object.__del).

Looking at the traceback though, it seems that the error is happening in gevent itself, that's why I doubt we'll able to fix this on Django's side. Also note that these errors are more annoying than harmful.

#20903 also reported this (but without reproducible code).

comment:2 Changed 3 years ago by James Addison

New information: I have also seen this traceback, and I haven't even used the model with the PointField:

Exception ignored in: <bound method Point.__del__ of <Point object at 0x7fb1c9e55a28>>
Traceback (most recent call last):
  File "/home/vagrant/.virtualenvs/myproject/src/django/django/contrib/gis/geos/geometry.py", line 125, in __del__
  File "/home/vagrant/.virtualenvs/myproject/src/django/django/contrib/gis/geos/libgeos.py", line 156, in __call__
  File "/home/vagrant/.virtualenvs/myproject/src/django/django/contrib/gis/geos/libgeos.py", line 160, in get_func
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2222, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 2155, in _find_spec
TypeError: 'NoneType' object is not iterable

comment:3 Changed 3 years ago by Claude Paroz

Triage Stage: UnreviewedAccepted

I also got one of those exception today with Django 1.8.5/Python 3.4.

I think we should simply catch TypeError in the destructor methods and ignore in case of exceptions.

So instead of current:

if self._ptr and capi:
        capi.release_srs(self._ptr)

Let's write:

try:
    capi.release_srs(self._ptr)
except TypeError:
    pass

comment:4 Changed 3 years ago by Tim Graham

Version: 1.9b11.9

comment:5 Changed 3 years ago by Claude Paroz

Has patch: set

comment:6 Changed 3 years ago by Tim Graham

Triage Stage: AcceptedReady for checkin

comment:7 Changed 3 years ago by Claude Paroz <claude@…>

Resolution: fixed
Status: newclosed

In cd40d9e7:

Fixed #25657 -- Ignored exceptions when destroying geometry objects

Due to randomness of garbage collection with geometry objects, it's
easier to simply ignore AttributeError/TypeError generally raised when
parts of objects are already garbage-collected.
Thanks Sergey Fedoseev and Tim Graham for reviewing the patch.

Note: See TracTickets for help on using tickets.
Back to Top