LayerMapping.save fails for large polygon record

[Tyler Erickson]

When trying to load a shapefile using LayerMapping, the save method fails when it encounters a large record. I will attach a shapefile that causes this problem to the page.

Here is the error that is returned...

taericks@optiplexe:~/django_projects/django_nasa_fire_dss$ python manage.py shell
Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) 
Type "copyright", "credits" or "license" for more information.

IPython 0.8.1 -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: import load_data
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted
taericks@optiplexe:~/django_projects/django_nasa_fire_dss$ 

Here is the script for loading the data (load_data.py)

import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
from psycopg2 import IntegrityError
from django.contrib.gis.utils import mapping, LayerMapping, add_postgis_srs
from fccs.models import FccsPoints,FccsPolygons

fccs_polys = 'data/fccs/fccs_poly_orig_wgs84_problem.shp'

mapping2 = {'gridcode' : 'GRIDCODE',
            'area_km2' : 'AREA_KM2',
            'geometry' : 'POLYGON',
            }

layer2 = LayerMapping(FccsPolygons,fccs_polys,mapping2, transaction_mode='autocommit')
layer2.save(verbose=True, progress=True)

Here is the model file (fccs/models.py)

from django.contrib.gis.db import models

class FccsPolygons(models.Model):
    
    gridcode = models.IntegerField(help_text="FCCS code of the plygon region",)
    area_km2 = models.IntegerField(help_text="Area of the polygon in SQ kilometers",)

    # GeoDjango specific Polygon Field and GeoManager
    geometry = models.MultiPolygonField(srid=4326)
    # GeoManager, a subclass that adds a rich set of geospatial queryset methods
    objects = models.GeoManager()

    class Meta:
        verbose_name_plural = "FCCS Polygons"

[jbronn]

The data file may downloaded from:

​http://geodjango.org/fccs_poly_orig_wgs84_problem.zip

The problem may be reproduced without LayerMapping, or even use of models:

>>> from django.contrib.gis.gdal import DataSource
>>> ds = DataSource('fccs_poly_orig_wgs84_problem.shp')
>>> lyr = ds[0]
>>> feat = lyr[0]
Python(45141) malloc: *** mmap(size=135168) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
terminate called after throwing an instance of 'std::bad_alloc'
  what():  St9bad_alloc
Abort trap

[jbronn]

This is a manifestation of GDAL bug ​#2428, and is not a bug in LayerMapping. Once 1.5.3 is released, I may consider adding an option allows users to set the OGR_ORGANIZE_POLYGONS configuration option to avoid crashes on Polygons with an extremely large number of rings (this one has 33,313!).